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FOREWORD 



The National Center for Bilingual Research was created by the 
National- institute of Education (NIE) and Southwest Regional Laboratory 
for Educational Research and Development (SWRL) in November. 1979. The 
Center, which represents the first federally funded major research unit 
in bilingualism and bilingual schooling in the United States, conducts 
a range of activities, including research, dissemination, and collaborative 
activities in research, training, and technical assistance. 

As part of its activities, the Center conducted a pre-conference 
workshop in conjunction with the Ninth Annual Conference of the National 
Association for Bilingual Education. The workshop. "Language Proficiency 
Assessment: What Does That Mean?", was held at SWRL on April 19. 1980. 
This report was developed i'n an effort to record, some of the experiences 
and knowledge that were shared during the half-day workshop. 

The National Center for Bilingual Research anticipates sponsoring 
other workshops and conferences to provide more opportunities for 
researchers and practitioners to communicate and share their knowledge, 
experiences, and concerns. 



VTctor E. RodrTguez 
Workshop Coordinator 



INTRODUCTION 



"Language Proficiency Assessment: What Does That Mean?" was the 
title of a workshop held at SWRL Educational Research and Development in 
Los Alamitos, CA, on April 19, 1980. The workshop was co-sponsored by 
SWRL and by the National Center for Bilingual Research in conjunction with 
the pre-conference activities of the Ninth Annual International Bilingual 

BicuUural Educat ion /Conference of the National Association for Bilingual 

J 

Education. 

The workshop was designed to provide an opportunity for educational 
researchers and practitioners to share knowledge, experiences, and concerns 
related to one particular area of bilingual education: what language 
proficiency is and how one measures language proficiency.^ A primary focus 
of the workshop was to be the assessment of non-English languages. As 
planning progressed, however, it became apparent that, although bilingual 
educators are primarily concerned with assessing oral language proficiency, 
the implications of language proficiency in assessing writing and reading 
skills also had to be addressed. 

In coordinating the conference presentations, the National Center for 
Bilingual Research was fortunate in being able to draw upon its staff and 
the staff Resources of SWRL. which over the past 13 years has distinguished 
itself in conducting research in language ski 1 Is assessment and in develop- 
ing curriculum materials for bilingual students. 

Members of the SWRL staff were asked to provide a general overview 
of formal and informal assessment of language proficiency, and to discuss 
the assessment of reading and writing skills. The staff of the National 
Center for Bilingual Research addressed the issue of assessing non-English 
languages, focusing primarily on Spanish and on Korean. 
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William Russell, SWRL Member of the Professional Staff, prepared 
a presentation on "Formal, and Informal Assessment of Language Proficiency," 
based on his work with the SWRL project to develop a Student Placement 
System for Bilingual Programs. Laila F iege-Kol 1 mann , SWRL Member of the 
Pr^essional Staff, spoke on "Reading Assessment and the Bilingual Teacher" 
and drew on her experience with the development of the reading component 
of the Survey of Essential Skills Project for the Los Angeles Unified 
School District. Ann Humes' topic was, "Assessing English Literacy Skills: 
Writing." Ms. Humes, SWRL Member of the Professional Staff, has extensive 
experience with the assessment of literacy skills as a result of her work 
in the SWRL Proficiency Verification System, the LAUSD Survey of Essential 
Skills Project, and the Project to Develop a Student Placement System 
for Bi 1 ingual Programs . 

Bonita Ford, Member of the Center's Research Staff, worked on the 
development of SWRL's Resources for Assessing Language Proficiency in 
Spanish (RALPS) and on the Diagnostic Assessment System (DAS) for assessing 
both English and Spanish language proficiencies prior to joining the 
staff of the National Center for Bilingual Research. Her presentation was 
on "Some Considerations in Constructing and Administering Language 
Proficiency Tests." Kenneth Kim, Member of the Center's Research Staff, 
chose to discuss the problems that one might encounter when attempting 
to adapt existing language assessment instruments for use with Asian and 
non-lndoeuropean languages. His consulting experience with Los Angeles 
unified School District, ABC Unified School District, and the California 
State Department of Education, has convinced him that this practice is 
more often based on expediency than on sound linguistic principles and 
can cause problems with the interpretation of scores and, subsequently, with 
the classification of students. 
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Seventeen participants attended the half-day workshop. Nine 
states were represented and participants included Bilingual Program 
Coordinators, Resource Aides, Teachers, and Teacher Trainers. Jhe 
agenda for the workshop is reproduced on the following page. The report 
of the workshop includes papers which were developed subsequent to the 
workshop presentations and a workshop evaluation based on questionnaires 
completed by the participants. 
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NCBR/SWRL 
NABE Workshop 
April 19, 1980 



AGENDA 

NABE Pre-Conference Workshop 
"Language Proficiency Assessment--What Does That Mean?" 



9:30 - 9:^5 Coffee 

9:^5 - 10:00 Introduction; 

10:00 - 10:20 Session #1 : 



10:30 - I I :30 Session #2: 



Dr. Vfctor Rbdrfguez 

Associate Di recter 

National Center for Bilingual 



Research 



Assessing Oral English Language Use. 

Mr. Wi 1 1 iam Russel 1 

Member of the Professional Staff 

SWRL Educational Research S Development 

"Formal and Informal Evaluation of Oral 
English Language Skills" 

Assessing English Literacy Skills. 

Ms. Ann Humes 

Member of the Professional Staff 

SWRL Educational Research S Development 

"Assessing English Writing Skills" 

Dr. Laiia Fiege-Kol Inrwnn 

Member of the Professional Staff 

SWRL Educational Research S Development 

"Assessing English Reading Skills" 



n :30 - 11 :kS BREAK 

11:^5 - 12:^5 Session #3. Assessing Non-English Language Skills. 

Dr. Bonita Ford 

Associate Member of the Research Staff 
' National Center for Bilingual Research 

"Assessing Spanish Language Proficiency^' 

Dr. Kenneth Kong-On Kim 
Member of the Research Staff 
National Center for Bilingual Research 
"Adapting Language Assessment Instruments 
for Testing Asian Students" 

- 12:^5 ADJOURN--BUS returns to Convention Center 
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NCBR/SWRL 
NABE Workshop 
/ April 19, 1980 



Introduction 
NABE Pre-Conference Workshop 

I. Objectives 

A. Learn what language proficiency assessment is. 

1. Learn approaches which are used to assess language proficiency 

2. Learn why approaches work or not,, 

3. Learn about assessing non-English language proficiency. 

B. Plan to use the ideas, 

1. Apply them in your s i tuat ion 

2. Seek materials and people to learn more about approaches to 
assessing language proficiency, 

3. Seek additional training in the use of approaches mentioned. 

C. Be willing and able to share with others what was learned. 

I I . Reasons 

A. Share the ideas available from various sources which are usually 
not shared. 

B. Speed up the process of dissemination of research information 
and ideas, 

C. Go beyond talking about ideas and being aware of approaches 
to trying to apply them. 

III. Roles 

A, Be inquisitive and willing to share your experience. 

B, Take responsibility to make your involvement worthwhile. 
Ask yourself: How can I use this information? 
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FORMAL AND INFORMAL EVALUATION OF ORAL ENGLISH LANGUAGE SK'LLS 
NABE Workshop Presentation 

If 

Wi 11 lam Russel 1 



We have a wide range of bilingual educators present. A number of 
you identify yourselves as school level coordinators and resource 
teachers, some are classroom teachers, a few are paraprofess iona 1 s and 
at least one is a district level coordinator. I will attempt to address 
this presentation to your various needs, especially to the needs of 
those persons who work directly with students. 

The title of this workshop is ^'Language Assessment: What Does That 
Mean"? Since language assessment measures language proficiency, I will 
first address the question, "Language Proficiency: What Does That 
Mean"?, and then I will sugges.t some ways that language proficiency can 
be assessed. I will distinguish between formal and informal assessment 
of language proficiency and explain why I believe both are important. 
Throughout this presentation Mwill emphasize oral language assessment 
as contrasted with reading and writTng. 

Here at SWRL, we recently completed a large scale analysis of 
English proficiency to be used irt developing an assessment system for 
limited English proficiency (LFP) pupils. Figure 1 is a diagram of 
the English proficiency analysis produced by that project. . ^ 

The specific purpose of this analysts was to identify the English 
skills minimally necessary for functioning in an all-English language 
classroom. It was designed to be one of the tools that school districts 
and others could use for selecting or constructing tests to be used in 
the placement of LEP pupils. The analysis i^. included in a set of docu- 
ments cal led Resources for Developing £ Student P lacement System for 
Bi I ingual Programs , which wi 1 1 be m3 i led' to al 1 school districts in the 
United States that have Title VII bilingual program^. 
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Figure 1. Organization of the SWRL English Proficiency Analysis by Component, Area and Subarea, 
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You see in Figure 1 that the first cut we made in the SWRL English 
proficiency analysis was to divide it into Read ing , Wr i t i ng and Ora 1 
components of English proficiency. These in turn were divided into areas 
and these areas further divided into subareas. Separate tree-graphs for 
each of the three components are shown in Figures 2, 3> and k. 

The principal sources for^the Reading and Writing components were 
actual textbook series used in classrooms across the country. Page by 
page analyses of these texts, together with other Information about 
classroom practices, revealed the skills that children at particular 
levels are expected to know. But oral proficiency is not determined by 
formal instruction as is. reading and writing. So, in place of instruc- 
tional materials^, we consulted studies of actual classroom language 
behavior, psychol inguistic studies of school age native English develop- 
ment, and a SWRL stydy of vocabulary use- 
Now, let's look again at Figure ^, the diagram of the Oral component. 
At the lowest level of this figure, oral language proficiency is divided 
into twelve subareas. This analysis was derived from information extracted 
from the sources I just mentioned. 

Let's pursue the oral language proficiency analysis a step further 
than Figure Take j for example^ the Complex Sentences subarea , 1 1 was 
further divided into Relative Clauses and Verb Complements . Below this 
level of analysis, every box on the oral language diagram branches into 
actual pupil behaviors. These last two levels of analysis are not shown 
on Figure ^, but they are listed in the table shown as Figure 5. 

In reality there are an indefinite number of behaviors for every 
subdivision of the analysis, but, for assessment purposes, we are 
interested in only those few behaviors that are practically testable. 
Figure 5 shows the behaviors (skills) listed for the Complex Sentences 
subarea of the Sentence Structure area of the Oral Language components 
Figure 5 also shows the grade levels to which each of these skills is 
ass i gned . 
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Figure 2. Proficiency Analysis for reading. 
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Figure 3. Proficiency Analysis for writing, 
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Figure Proficiency Analysis for oral language, 
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(iride level for aswssmfni 



SENTENCE STRUCTURE (SS) 



Complex Sentences (cs) 



Relative Clauses 

1 100c Comprehends relative clauses: relative pronoun 
as subject 

1 lOOp Produces relative clauses: relative pronoun as 
subject 

1200c Comprehends relative clauses: relative pronoun 
as object 

1200p Produces relative clauses: relative pronoun as 
object 

1300c Comprehends relative clauses: relative pronoun 
omitted 

1300p Produces relative clauses: relative pronoun 
omitted 

1400c Comprehends relative cUuses: relative pronoun 
whose 

1500c Comprehends relative clauses: relative pronoun 
preceded by a preposition 



Verb Complements 



3200c Comprehends the presupposed truth of factive 
clauses 

3300c Comprehends the roles of the participants in 

sentences with promise followed by an infmitive 
phrase 

3400c Comprehends the roles of the participants in 
sentences with jawy. hard, fun followed by an 
infinitive phrase ' 



Figure 5. A Portion of the Oral Language Analysis at the Skills Level 
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Each of the skills in the SWRL analysis Is extensively specified. 
For example, let's look at just one of the skills from Figure 5, skill 
number 1200p, "Produces relative clauses: relative pronoun as object." 
The full specification of this skill is shown in Figure 6. Tlie best way 
to understand a skill in this analysis is to look at actual examples of 
the behavior in question. Such examples are given in the full specifi- 
cation shown here. Also, advice is given here about how to assess this 
skill. 

Up to this point, I have briefly shown how the SWRL analysis of 
English proficiency progresses from a first cut into Reading, Writing 
and Oral Language until the actual skills level Is reached. We have 
been looking at a small sample of the analysis of English proficiency 
which was done at SWRL for the Resources for Developing a Student Place- 
ment Sj^stem for Bilingual Programs. These Resources provide a substantial 
basis for the selection or construction of tests to assess the English 
proficiency of LEP pupils. Similar analyses could be done for other 
languages, but I do not know of any. Because of the need for such 
proficiency frameworks for other languages, there may be a temptation 
to translate some of the assessment items that appear in the Resources, 
but this would be a serious mistake. You have seen the many levels of 
analysis that were used to organ lize - the skills of the Resources. A 
similar analytic framework for another language would be quite different, 
and the skills that it would subsume would only occas ionally be similar 
to those for Engl ish. 

As an example of an assessment Item in another language, let me 

show you two relative clause skills in Spanish together with^tasks for 

assessing proficiency in per-forming these t\cio skills. These' materials 
arc shown in Figure 7« ' 

For the Spanish examples, I purposely chose skills that are very 
similar to the English skills we just looked at, so that the Spanish 
examples would be more understandable, but keep in mind the caveat 
regarding translating skills. Even these very similar skills relate 
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00 SScs IZOOp (continued) ' 

To *a«« orocesslng requirements at this level, the relative clause !s 
located at the end of the sentence; i.e.. nidifying elther.a d.rect 
object or a predicate noun; e.g.: 

Direct object: Mary caught the ball that Jim threw. 
Predicate noun: This is the ball that Jim threw. 

In addition, at this level, the relative, clause is restrictive (not set 
off by pauses) . 

Relative pronouns that can be used with this type^of relative clause are 
that, which , who/whom. Not all are reconmended , however, for the reasons 
enumerated below. 

a) That ; That Is the most commonly used pronoun In this type of 
T^TItiveTTause. That can be used with either human or non- 
human nouns although use, with non-human nouns is more common. 



I saw the ball that John hit. 
I saw the boy that the dog bit. 

b) Which: Which is used only with non-human nouns in the modified 
"I^SJiTphraseT Some authorities do not approve of its use in 
restrictive clauses. 

I saw the ball which John hit. 

c) Who/whom: This type of relative clause takes the whom form 
"wTTtTng and ve^y formal speech; e.g.. I saw the boy whom 
the dog bit. However, who is cormonly used, especially irr 
speech. Students should not be expected to discriminate 
usage of who/whom . 

For production, students may use any of these pronouns. Although which 
should be distinguished from who /whom, students should not be required 
ro use whom rather than who . 

Assessment 

Sample Items 

1. A. [picture: gfrl, wrench In hand, standing next to 

bicycle she has Just fixed] 

Janet fixed tha blcyda. 



Figure 6 (page 2) 
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00 SScs 1200p (continued) 



2. 



B. (picture: boy, wrench In hand, standing next to 
bicycle he has just fixed] 

Tom fixed the bicycle. 
(Point to picture.) Which bicycle is this? 

A. [picture: man standing next to tree he has chopped 

down] 

The man chopped down the tree. 

B. [picture: woman standing next to tree she chopped 

down] 

The woman chopped down the tree." 

(Point to picture.) Which tree is this? 

A. -[picture: monster standing next to a chair *vith a 
large bite taken out of the chair] 

The monster ate this chair. 

[picture: a monster sitting on a chair that is 
crushed by its weight] 



B. 



The monster sat on this chair. 
(Point to picture.) Which chair is this? 
item Description ♦ , 

The exami^ner displays" two pictures and, pointing to each picture, 
says the corresponding descriptive sentence. Then the examiner 
points to one of the pictures and asks, 'Vh.ch is this? 

The stud ent res ponds orally with an answer containing a relative 
clause. (If desired, the examiner may then point to and query the 
other picture for an additional student response.) 

Each pair of pictures depicts the same object acted upon by two 
different people or animals. The two objects are distinguishable 
solely on the basis of who or what Is acting on them. 

The sentences describing the pictures are in the form desired for 
the relative clause to be elicited; e.g., "The monster ate this 
chair," is equivalent to "that the monster ate." 



Figure 6 (page 3) 
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00 SScs I200p (continued) 

Th«r« tr* • number of possible responses for each Item, all -iss 
containing relative clauses; for example, the follcwlng (plus 
other variations) should be considered correct for Item I: 

This Is the bicycle that Janet fixed. 
This is the one that Janet fixed. 
The bicycle that Janet fixed. 
The one that Janetj^^^lxed. 

Moreover, the student need not describe the picture exactly as the 
examiner does, so long as an appropriate relative clause Is used; 
e.g., the one that Janet has. Any appropriate relative pronouif 
(I.e., which, who; see Skill Description) may be used Instead of 
the t . 

Cornnent: A problem with assessing this skill Is that the 
student can give an accurate response without 
using a relative clause; e.g., the one on the 
bicycle, the one with the bicycle. Or the student 
may use a relative clause In which the relative 
pronoun Is not the object; e.g., the one that was 
eaten by the monster, the one that has a hungry 
monster. Moreover, the student may use the appro- 
priate relative clause without a relative pronoun; 
e.g., the one Janet fixed. (Use of this more complex 
structure is assessed by Skill 02 SScs 1300p.) If 
the student seems to understand the pictures and the 
questions, but does not respond with the appropriate 
relative clause, two other approaches are possible: 
modeling and imitation. 

Modeling . For modeling, the examiner supplies the 
appropriate response for a few items and then asks 
the student to respond to others; e.g.: 

A. Janet fixed the bicycle. 

B. Tom fixed the bicycle. ' 
(Point to picture A.) Which bicycle is this? 
This Is the bicycle that Janet fixed. 

Imitation . If modeling does not produce the desired 
response, the examiner may have the student Imitate 
his/her use of the correct structure; e.g.: 

I'm going to tell you about these pictures. 
When I'm done, you wi?l say what I say. 

A. This is the bicycle that Janet fixed. 

B. This is the bicycle that Tom fixed. 
(Point to picture A.) What Is this picture? 



Figure 6 (page k) 
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di fferent functions 



language^ systems quite 
in each language. 



differently and carry somewhat 



Notice in Figure 7 that the pupil does not respond exactly as 
expected, especially to the second item.^ The last paragraph of Figure 7 
makes the poing. that' i i is the successful production of the relative 
clause fonr, that is being elicited, so the reponse counts as correct 
eyen though vocabulary/ style, dialect or other features of the response 
may not be expected shcool usage. One must be clear about whatjs being 
tested and then evaluate responses on that basis. An analytical frame- 
work like the one used in the Resources is helpful in making clear exactly 
what category of behavior a particular skill belongs to, and therefore, 
exactly what is being, tested by , the assessment items that test this skill. 

So far, we hav.e been looking at a basis for making fqrmal assessments 
of language proficiency. But, as a person experienced' in working with 
bilingual p'upils, you are in a position to make educated informal, judge- 
. ments as well as administer formal tests.- Such informal judgements are 
most sensitive to "global" aspects of successful communication as distin- 
guised from the discrete aspects that were sorted out by the SWRL analysis. 

In the present state of the art of language assessment, I belive ,that you . 
can do the best job of placing LEP pupils by using a balance of experienced 

intuition and formal language assessment to capture both global and 
discrete aspects of communication behavior. 

Let me suggest in conclusion two areas of caution when informally 
Judging the language proficiencies of a LEP pupil. With these cautions 
in mind, your experience with LEP pupi Is "should produce excellent 
intuitions about the "global" language proficiencies of these pupils. 



.14- 

" Inti-rvifw sfSsloiis consisted of tlic prescnta.tion of n series 
of pictures to ilu- pupils.. Kach picture contained two frames, with 
two animal characters in each: a cat and' a bird or a cat and a 
d6g. After describing the situations, the Interviewer asked the 
child to dist.inguish one cat from the other by the question: 
"iCual gatq es estef*- Interview It^ffiB^re-divided evenly i-n*e--Ew0 
basic situation types according to the subject-object" relation of 
the characters. An example of each of these types is given below, 
along with the elicitation procedure and typical pupil responses. 



Item Type I 





(a) 



(b) 



Interviewer; 



(Points to the cat in frame (a)) 

•'Este gato esti persiguiendo a un pajaro." 

^ I. 

(Points to the cat in frame (b)) 
"Pero este gato no eSta persiguiendo al pljaro." 
(Points to one of the cats as specified in the 
written script, in this example the cat in 
frame (a)) 

".iCual gato es este?" *• 
Pupil: "Este es el gato que 'sta persiguiendo el pajaro." 
Interviewer: "StI bion. Este es el gato que esta persiguiendo al 
. pSiaro. Dfmelo otra vez." 

^ II " 

Pupil:'^ "Kste es el gato que '.sta pe,rsiguien do al pajaro. 
Figure 7 Two examples of assessment .items (production of rel . 'cl'ause 
FROM: RUSSELL, WM. AND DAVID SNOW, SWRL TN-2-76-08, 1976' ' 
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(a) 

Interviewer : 



Pupil: 
Interviewer : 



(b) 

(Points to the cat in frame (a)) 

'Un perro esta empujando a este gato." 

(Points to the cat in frame (b)) 

'El perro no esta empujando a este gato." 

(Points again to one of the cats, say, the 

one in frame (b)) 

'iCual gato es este?" 

'Este es el gato qu'el perro no lo anda'mpujando/* 

•Muy bien. Dimelo otra vez: Este es el gato que 
el perro no esta empujando/* 

Pupil:' Este es el gato qu'el perro no lo anda'mpujando.'* 

"Eacli interview was preteded by a practice session, the purpose 
o\ w.hiih was in familiarize the pupil with the situations, characters and 
vv^•abula^y items, and to teach and practice the task itself. Throughout, 
insLructions were repeated, or the task reviewed, if hesitation on the 
pupil's part indicated that additional support would be helpful or if 
his response failed to ^ive the interviewer confidence that the task had 
been understood. Our purpose was to make the task as clear as possible^ 
hoth in order to enable the pupil to experience success In performing it, 
and to encourage an appropriate response with a relative clause, the forni 
of such responses being what we intended to observe. 

Figure 7 (continued) 
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CAUTIONARY AREA 1: DIALECT 



It is easy to confuse the use of non-standard dialect forms with 
limited language proficiency. Suppose a ch il d says in English., 

■'She ain't got none," 



or in Spanish, 

"Tu vites come el troque andaba asina." 

Because these utterances contain non-standard dialect forms, they are 
inappropriate in certain social circumstances. But this inappropr lateness 
Is not the primary concern of bilingual programs. A bilingual program 
must first consider the pupil's proficiency in a given language without 
regard to dialect. It is possible to be quite proficient in a language 
without being proficient in a particular dialect. The confusion between 
dialect appropriateness and language proficiency is encouraged by the 
common perception of non-standard forms as "errors" or "bad grammar." 
These forms may properly be viewed as errors of social appropriateness 
in particular situations, but they do not reflect a lack of general 
proficiency in the language. 

CAUTIONARY AREA 2: INDIVIDUAL VARIATION 

Another caution in Judging proficiency has to do with individual 
variation in native proficiency. This situation is particularly difficult 
for the bilingual educator for reasons I will attempt to set forth. I 
bring up this difficult topic here because of its ser ious impl icat ions 
for the welfare of bilingual childrea. 
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The main points of this cautionary area are: 

1. There is a basic aspect of language proficiency that does not vary 
much across mature native or near-native speakers of the language. It 
follows that all normal speakers have an intact native language. The 
observal^ion that some bilingual speakers find themselves ''between 
languages*' is probably an illusion for reasons I will give. 

2. Basic proficiency in a second language does, of course, vary from 
speaker to speaker up to the stage of acquisition where the second 
language is near-native, then basic proficiency behaves as in native 
speakers. 

3. Basic proficiency may improve with instruction in a second language, 
and so it is taught in bilingual programs. In mainstream classrooms, a 
different kind of proficiency (which I ca 1 r"vi rtuos i ty") is the subject 
of instruction. 

For assessment, bilingual educators must understand which kind of 

1 

proficiency they are evaluating. 

1 

I 

The way native language proficiency varies from user to user is best 
understood by positing two kinds of native proficiency, "basic proficiency' 
and "virtuosity.*' Basic proficiency includes control over the sound 
systemjand syntactic patterns of the language, while virtuosity includes 
such things as skillful use of the language to convince and be convinced, 
explain and understand, teach and learn, entertain and be entertained, 
etc. "Basic proficiency" and"v i r tuos i ty" are my terms, not terms from 
the litjerature. In fact, this area of research is sufficiently unexplored 
at this time that the following comments should be taken merely as 
convictions based on years of research experience rather than as 
establ ished facts . 
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Basic proficiency does not vary significantly from native speaker 
to native speaker at any given stage of language development. That is 
to say, all normal children pass through similar stages of basic profi- 
ciency development in their native language and eventually reach an adult 
level that is approximately the same as that of other speakers. 

If basic proficiency is a characteristic of all native speakers, 
then it follows that al' normal bilingual children have full basic 
proficiency in at least one language. It is often contended that some 
bi I ingual s are "between languages" and somehow do not have a native 
language in the way that mono'l ingual speakers do. I believe that this 
conclusion results from a confusion of basic proficiency with virtuosjty.. 
For instance, vocabulary development beyond the words needed for very 
basic communication Is a matter of individual virtuosity and is not 
necessarily an indicator of basic proficiency. 

It is important when assessing the language proficiencies of bilingual 
children to identify the languages in which they have basic proficiency, 
regardless of the virtuosity that they might have in these languages, 
because other proficiencies will be learned differently by individual 
children, depending on their potential for skillful use of language. 
We must first identify the languages in which the- basis for this skillful 
use of language has been acquired. Then we can judge what more can be 
expected of a pupil according to his or her potential for virtuosity. 

For assessment, one important implication of the 'dist inct ion 
between basic proficiency and virtuosity is that there will be one of 
more basic language proficiencies for each child, and these must be 
ident i f ied . 

For instruction, the virtuosity versus basic proficiency distinction 
has a further implication. Basic proficiency is typically acquired with- 
out instruction. Instruction may facilitate its acquisition in a second 
language by focusing experience on the new language. Beyond this, 
however, instruction will focus on virtuosity. 
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As a bilingual educator, you are much more concerned with basic 
proficiency than other educators. And it is more important for you to 
be aware of the distinction between basic proficiency and virtuosity. 
For instance, you may place a child in a mainstream classroom because 
his or her basic proficiency in English is native-like even though the- 
child's evidence of virtuosity in English i,s not great. A reason for 
doing this would be that you judge that this child's particular 
potential for virtuosity has been realized at this stage of development. 
On the other hand, you may retain another child who has the same basic 
English proficiency in a bilingual classroom so as to allow the child's 
great potential for skillful use of English to develop before the pupil 
is placed in competition with .native speakers. The latter decision would 
have to assume that the bilingual classroom in question offers rich 
experiences in English accompani<:d by opportunities to succeed academically 
in a native language. 

It is even more important to your pupils than to pupils of the 
mainstream for language assessment to be skillfully performed. This is 
just one of the many ways in which your job is more difficult. 
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ASSESSING ENGLISH LITERACY SKILLS: WRITING 

A goal of transitional bilingual programs is to provide students with 
sufficient proficiency in English to function in the monolingual class- 
room. This paper discusses the steps involved in describing the English 
writing skills required in grades 1-6. the assessment of those skills, 
and some of the problems that were encountered in accomplishing the task. 

SKILLS IDENTIFICATION 

The first task involved identifying the critical writing skills, 
and the first problems were encountered when the literature was reviewed 
for help: 

1. Although there currently is an intense interest in writing, 
the literature does not now provide adequate information on 
the specificity or range of critical writing skills. The lists 
of skills suggested by authorities are usually too brief and the 
skills themselves are too general, as these are. for example: 

Uses accepted punctuation and capitalization. 

Uses accepted form and appropriate language in varying 
types of written communication (Petty. Petty. Newman, 
and Skeen. in Squire. 1977. PP- 89-90). 

Even more general is the frequently cited. "Writes effectively." 

2. Sometimes a listed "skill" really consists of a skill plus 

a task or an item description.' The following skill statement 
is typical : 

Writes a description of a picture of a painting. 
"Writes a description" is the skill; the rest of the statement 
refers to the specific object to be described. However, the 
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same writing skill is often relisted as a separate skill that 
is differentiated only by that objectto be descr i bed--for 
example. "Writes a description of a person." 
3. Skill identification Is also hampered by the literature's 
failure to designate the discourse-type subskills of a skill 
statement such as 'Vrites a description." that is. subskills 
like "Uses sensory terms." "Uses spatial ordering." 
A. Finally, some skills proposed In the literature are more than 
too general; they are vague. This skill sampled fr«. a state 
competency document (Georgia Department of Education. 1968) is 
typical: 

understands that writing is a tool of communication 
(p. 77). 

The CollelfMon of Skills 

Despit. thes. probl.ms. "a.y skills w.r. Id.rtifi.d during the 
..vi.w of scholarly r.co,™«ndat iors ard compeUrcy lists. Each broad 
skill, such as "Writes a description," was analyzed to identify its 
subskills. The collection of skills and subskills was then compared 
with skills. derived fron, the SWRL data available on language-arts text- 
hook series, and ,ny duplicated skills were deleted. To collect the data, 
page-by-page task analysis Is performed on textbooks to determine, the 
content of the instruction and the manner in which the content is practiced 
computer processing of this data produces lists of content by grade lev.,. 
Conse,uently. a large data base describes actual Wtbook (and therefore 

Classroom) Instruction in detail. Pag. 2 of t dout is., copy of an 

actual computer printout of the textbook data. 
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The Critical Skills 

~ / 

The comprehenJve list was then screened for essential skills 
according'^to three|criteria: appearance on state or local competency lists, 
textbook emphasis. Lnd the nature of a skill as either a prerequisite of 
skills at a higher grade level or as a subskill of another skill. Skills 
that met these criteria were retained on the list with the exception of 
some that were covered by the reading component (such as dictionary skills). 
The list was consequently reduced to approximately 180 different skills. 

GRADE LEVELS 

The next step in the project involved assigning appropriate grade 
levels to the collected skills. Again the literature posed problems: 

1. Some language arts authorities do not assign grade levels to 
skills because they feel that sequencing is the local school 
district's prerogative. 

2. Some authorities designate cuts in sequencing in blocks such 
as K-3 rather than in grades. 

Other sources list skills for more than one grade level without 
noting the criteria that differentiate performance among grade 
levels. These grade 3.^. and 5 skills are such an example: 
Grade 3: Writes short stories and/or poetry (p. S'*)- 
Grade 1»: Writes stories and poetry (p. 68). 
Grade 5: Writes stories, plays, and poetry (p. 92). 

O (Los Angeles City Schools, IBS'*) 

' Still other sources sequence skills objectives without presenting 
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,„y .vid.r« to v.lid.» their ordering, and frequentiy their 
sequencing confilcts with textbook sequencing. 
The chert on pege 3 of the handout di.p.ays the grade-ieve, variation 
.h,t occurs between authorities' designations and textbook presentation. 
The chart is based on a stud, by Goiub (i97i), who anaiyzed differences 
.etween grade-ieve, designations found in iang.age arts books for teachers 
end the grade ieveis of instruction in students' textbooks. The chart 
Shows the grade-ieve- consensus found in four books written by eminent 
language arts schoi.rs. the «d,i grade ieve, at which four textbook 
series presented each skiii. and the grade-ieve, designations suggested 
,y .he textbook data. The SWRL grade ieve, is the one at which »»st 

or usua,i, ai, analyzed textbook series present (or have presented) the 
s.n.. This designation ™ay or .ay not be the ^dai ieve,,; usua.iy 



it is not. 



Considerabie grade-,evei tion is also found across state and 

handout. This chart i„ustrates t iation across s tates. 

information on these ski i is was complied fro™ representative state 
.ocu^ents (Lawior. ,375), which frequently piace skl„s at ,eve,s higher 
.ban those at which the ski„s appear in textbook instruction. 

Because the grade levels reported In the literat d by the 

competency lists are inconsistent and often do not parallel the actual 
l..el of textbook Instruction, the grade levels of the writing sklHs 

.he ™>st reliable data available on the classroom activities across the 
entire county are the textbooks students use. And when students must 
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function In the English classroon,. the actual activities occurring there 
are the important criteria of success In that classroom. However, when 
an Individual school district Is developing its own list of critical 
skills, the grade levels of the skills, as well as the criticaV skills 
themselves, should be determined on the basis of the textbooks and 
curriculum of that particular district. 

THE SKILLS FRAMEWORK 

After the grade levels were specified, the skills were organized by 
categories into a framework of writing skills. The major categories of 
that framework are Handwriting, Spelling, Mechanics, Language, General 
Discourse, and Discourse Products. A Discourse Product is defined here 
as a whole composition, such as a story, an essay, a poem. Composing 
effective written products requires the use of many skills, or subskills, 
and these are covered in the General Discourse area. Page 5 of the 
handout shows a portion of the detailed f ramework-speci f ic ski lis listed 
for subcategories of the General Discourse category. 

The framework presents one way to organize writing Jkills. Other 
frameworks are possible; however, this one was satisfactory for our 
purposes. For Identification purposes, category letters and Individual 
skill numbers are combined to give each skill a code. 

SKILL SPECIFICATIONS 

After the skills were organized into a framework, skill specifications 
were written to del ineate each ski 1 1 and to ensure its appropriate 
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.,s«s«»nt. The skill sp.ciMcatlons .r. based on content and task 

ana.vsis - """" """"" 

referenced in preparing the ski., specifications. Sa.p..s of complete 

specifications for specific ski.Is wi.l be passed out later. 
Skill Statement 

The skill statement appears at the top of the page after its code 
number. ,n general , .he ski 1 1 state^nts are cast in ter^s used both 

™ny language arts authorities and by competency lists. These sk.ll 
st^te^ents generally must be «K>re precisely defined by further explication 
before assessments can be devised. 

Skill Descriptions 

This explication is accomplished by the Skill Description. It 

notes factors Involved in the use of each skill. Including ot t 

sl<ills and prerequisite l<nowledge. 

Assessments 

The Skill Description Is followed by the section on assessment. For 

^.t Skills, three or ™re sample assessment items were written. How 

,„r t ting samples that assess Discourse Products, only one Ite. . 

was written for each skill because these Items are complex and because 

only one writing sample per skill should be Included on an actual assessment 

instrument. Included In wri tlng-sample specifications .,e the prompt, the 

scoring key. and scoring guidelines. , 

.r» the most appropriate type for the 
The Items that were written are the most 

► u,* r.niilre a written response were 
skill to be assessexJ. Items that require a wriL e k 
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frequently provided because they are often the i«st appropriate 
Issessn^nt type. They were the only type included for Discourse Products 
because a student's ability to compose a complete wr i tten product can only 

b d through an actual composing task. When wri tten- response it.ms 

were suggested for skills other than Discourse Products, mul tipl^^-choice 

Uem also included so that they can be used .s substitut 

administering wr i tten-response items is not feasible. When multiple-choice 

Uem It PPropriate as or n«re appropriate than wri tten-response 

items. onVy mul ti pie-choice " i tems were prepared. 

Item Descriptions 

Following the sampl ment items is an item description that 

provides guidelines for preparing additional ment items that are 

precise and appropriate, u 

PROBLEMS 

prcM.™s that wr. ..coun„r.d .n wr .n, th. s.U. sp.c.f .cations 
.ncountered others who wish to d.v.op their own s.iH specifics- . 
,,P„. hased on their own ioca, curriOu™. These pro.,..s are in addition 
„ the usua. or,.s of avoiding sexis™. ethnic stereotypes, and violence. 

one problen, was the discrepancy between the simplicity of the skill 
state^nt and the complexity of the actual skill. The skill statement 
fr.,uently nan«d what see^d to be a straightforward skill. However, 
writing the skill description required close analysis of all factor, 
,„H.,ent In the skill and relevant to its asses,n»nt, and this close 

.„„,,is oft I.d that the skill was far more complex than It, skill 

state^n. suggested. So™ skills were exposed as so complex that they 
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should not be assessed. For example, many competency lists include for 
assessment a handwriting skill, "Writes sentences from dSctation." 
However, content and task analysis revealed that this is no simple 
handwriting skill. Writing sentences fVom dictation involves nany other 
skills, such as those of spell ins, mechanics, and listening. 

We found that some skills can only be assessed in terms of anot'fer 
skin. Distinguishing' between proper and common nouns is one such skill. 
It is assessed by capitalization skills, Identifying most sentence types 
is covered by assessment of punctuation skills. ) 

Sometimes a skill's complexity wasn't fully realized until 
assessment items and their descriptions were written. Then the skill 
description (and often even the items themselves) had to be completely 

rewritten. , 

Directions also posed a problem because a single i nappro^jr iate or 
omitted word may mislead students. Specific directions must be 
painstakingly written so, students unquestionably understand what they 
are to do. Yet the directions must be devoid of technical words, when 
this restriction is .possible. / 

Care must also be taken that each item assesses ^ly one skill. 
For example, items assessing ability to construct c/ordlnate subject-noun 
phrases must not include assessment of the skill o> subject-verb agreement 

Written-response items and multiple-choice items pose different 
problems. Written-response items often appear-to be simple to devise, 
but this simplicity is deceptive. Such itemsmust be carefully developed 
or they will not elicit the desired response/ If, for example, the skill 
being assessed is the transformation of a declarative sentence into a 
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,„„io-.. t.h. ...n, .ust b. d.vis.d so that th. stud.nt does not simply 
r.„rit. th. seoteoc. in th. sam. word ord.r and add a qu.stion ..ark. 
Also, any compon.nt skil.. or subskill. that should b. ass.ss.d .by th. 
it.n, ™ust b. .xplicitiy prompt.d. If, for .»™pU. precise language is 
to b,e used in composing a written product, the it.™ stimulus must t.ll 

Students to use *'exact words.'' : « 

Written-response items are also difficult to score, so guidelines 
for rating responses must be included in the skill specifications. If 
such rating criteria cannot be devised, this i nabi 1 i ty suggests that 
there is a deeper problem with either the ski 11 description or the 

assessment item. ^ 

On the other hand, multiple-choice items are easy to score, but 
they are usually even more exacting to devise than are the written- 
response items. Multiple-choice assessment items are particularly 
difficult to devise for writing because they should require a response 
t^at assesses writing ability rather than reading ability. This is an 
important factor In constructing writing assessment items, but it is 

frequently ignored. 

Th. r.ading r.,uir«n.nt of som. mul t ipi.-choice items must also 
be considered. Th. amount of r.adi ng students must do to select a 
corr.ct answ.r should be as limit.d as possib..; however, this r.ading 
reguirement so^ti^s cannot b. .9w.r.d and sti.l have the item assess 
th. skil...-Afso the reading vocabulary used in the items must b. limited 
to those words that ar. on contro.l.d vocabulary lists for .ach grade 
,.v.l. writing a good prlmary-l.v.l it.m with only those words 
available on primary reading lists is challenging. 
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Th. 'construction of distr.ctor choices for .u.tip.e-cho.ce lte™s 
3,so poses d.v.,op™nt problems. DIstractors .s.d .n the sa™p.e lte™s 
a.scr.b.d .n th. It.™ descriptions sho.-d be designed and wrUt.. 
so that tbev serve diagnostic purposes, and this diagnostic Information 
Should be discussed In the Ite. descriptions. Tuther^re, distractor 
choices .ust be carefuUy screened so that the, are appropriate for the 
,rade level of the S.IU being assessed; for example, although the 

, u an appropriate distractor In Ite^s fpr spel , Ing - 
IS assessed at grade 5. £h Is not an appropriate distractor for I 

when It Is assessed at grade I. 

Developing a language skills framework Is a t,.e consuming task. 

wrtMna Skill specifications is even more 
As should be evident by now, writing Ski u sp 

„„.-consu.lng.. Vet such speclflcatio t be written to determine 

;.„at IS to be done and to validate what has been done. The a.l-too- 
frequent approach consists of reading the S.IU statement and then 

Whipping off a few Items with a correct several distr.actors 

without regard to Isolating a single skill for assessment, to considering 
..e «.de being assessed, to guaranteeing the precision of the directions, 
to serving diagnostic purposes with the use of appropriate d.stractors. 
writing Skill specifications for this project required the fui.-tlme, 
.ight-hour daily effort of four well trained professional staff members 

,„r approximate,, two ^nths. How this kind of effort is essentia, 

wHen t , ts of a project may be used In determining bl I ingua, 

student's' potential reclassification. 
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SPECIFICATION" 



I OH A HICMANICS SKILL 



Ski M r Wli «p 7110 
SkjJlJlSiSKlli 

U».s .postroph. in singular poss«*Iv. fonns. 
tmi Pa^crlptlow 

Th. ,tud.nt -u.t ^t^Un U« root -ord and the s-. e.g., 

the .Ingul.r P?*""':' [ C^I'^^for-ed from sInguUr proper or 
boy's) .. SlnouUr PO""^'^" T?) or frop Indefinite pronoun* (e.g.. 
TSSSn noun* (e.g.. i2$!l^'|3f^' "Ll^^ «t Included In assessment 
someone's) . «»**>'-^^^lHIL^'„r^"U « nouns In te-tbook Instruc- 
ISFTBiMuse they ere not "^r!""!*," ,„„,^te noun* ere not assessed 

at thi. 9r««e level. l^^l^uir.nd .re not .ceepted by 
for ^Msslve. .Inc. t>^/: L':^.7:::5,. but - 
soM author It! •» (a-g-f ••^ cn» ^ 

.w.icit in this .kill IS •'>'''i:.r ''i:i:rr!';t5urt:\'hi:M 

Sll "ouns .nd •""Viyb:^:^:* rnjul.^^d iur.l poss.sslves at 
be e»P.ct.<l to dlstlngu,l.h b.t*<.«» smgu 
this gr«l. I.V.I. 

IwTpraraquisUe to this skill. 
A^sasswant 

<Mm ^\m Items 

Directions: Which word Is right? 

, uncia is a see captain. 

' A. Olck's 

B. Olcks 

C. Olc'ks 

2. Betsy h.. . P.t O'^-- She wants to clean the 
eg.- 
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A. bird's 

B. birds 

C. bir'ds 

3. All the 

A. boy's 

B. boys 

C. bo'ys 



went down the stairs. 



Oi rcct Iv>ni : 



r;n in the b^^nk with the correct form of the .-^o^r 
word. Tne form shows ownership. 



1 ined 



k. Tony has a bicycle. 

bicycle is red. 

5. The b i rd has a nest. 

The ^* • 

6. My sister has a kitten. 

^ ^ kitten drinks milk. 



ifmm Description 
responses. 

The stimulus 'or the constructj^d-respons. U« fVP-^<i;«' J'f > '« 
, sertence with an underlined the bl.nk 

(e.g., James's or James* ) > 

Proy.Ui ''•c'-l-"-/rrruU! "a'l"..Tn"t«T' 

particularly when ""^.^i^ytt ncluded so that the student 

the sentence "Betsy h.s . P"* The sentence "She 

knows il..it the possessive form ''^''^^d^j „ot provide enough 
W4MI-. I., clean t*^... . , corre ct choice sine. .lth.r bl^dS or 
ItiloriiMllon to faclTTtat. . correti 

birds' (ould be correct. 
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srccincATior, ion a gcmchal t>\Hmsi skill 



Skill ; W* fiO« 1120 
Skill Sffwnt 

CUborafs paragraphs. Ll«Us • p-ra^raph to ona n-in idea. 
^kill Docripttoo 

pr*r«qulslt« to thit skill. 

.bill Is b«tt asMtswi by a »#rltln« SMpU. I' 

Itll J^rbTuM*. Car. .ust b. t.Mn that tha Ita. 
tYM *-t «»t a rMdIng rathar than a 

In a*MtMmt Inttrui-ntt for writing. JJ'Vl*"V 

raqulrat ttydant. to r«d a l«J»9r«ph 
Uuet a t«.t«nca that 9lv«. tha .aln ^ 

thwi • fro4»ictlv«-l«n«|Mafl« ihin. 
SawDia I taw 

Olractlont: On. .antanea can b. addad to »•» 

" It fits tha Idaa of th. paragraph. Which »ant«ica 

It It7 

1 Football pl.y«r« mt%t ba ttrong to play In a 

omm Thiv «u.t ba .bla to knock othar pUyart 
^\ !L.t b. abia to run '"t,^""" 

7Z*r* can carry tha ball to Mka point. In tha 

i. Foocb«ll playart waar thlrtt with numbars on 

C. IS" football playar. 90 to .chooJ to laarn to 

pUv ban. 
p. lasketball playar* run fait. 
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W<i GDa (continued) 



You must take good care of a pet b.rd. toy 
must give it water to drink. Its cege must ba k^ 
clean so the bird will stay well. You can put p^r 
In tha bottom of the cage to Mke it easy to cl#an. 

A. Tha bird will get sick if U <>oes not have -tar. 

B. A pet bird can sing sweatlyw 

C. You cen get a p«t bird at a ^t store. 

0, A pat rabbit iMy also live in ^ caoa- 

The trunks of elephants are ftranta, kut useful. 
Elephants use their trunks to pick. up thinfs. 
Food is one thing that elephants pick up with 
their trunks. They also use their trunks to 
splash water over themselves. 

A. Elephants use their trunks to put food iV tha'r 
fliouths. 

1. Elephants hav« funny feat, too. . 

C. Tha ears of elephants have hair on tna«. 
0! figs also look stranfa. 



itmm Description 

The stimulus paragraph has an initial topic ••"jr^lJ^l^^trtld 
IkI «min Idea All Other santances pertain to tha main 
the main .^.^ ^ of these santmncas should 

in that topic sentence. "I? ^ .-nt^ica should state 

aipoiltory dl.courw. Th.r.for., th. ttl-lu. I. *<rltt«. !•» 
•xpotltory dlicowria. 

Th. .tu-ant .alact. ^-^^^ r;2i;^;;ou^;^'':Uh:r^^^ 
fits th. win IdM. Th. corraet im»-.r t " , ,„ 



\ 



5i 

1 



II 



Fruit and M »*»pp>^ Cr««jn> Tr— t 



I cup of ftr«wb«rri«f 

1 cup of cut-up bananas 

2 copf of whipped craai^ 
I charry 



"llJ^JTlfi cat Ions for pictura 
drawn: 



Fwhtppad 



charry 
c raaw^ 



t banana s Ucc* I 

I whlPf *** craam L 
f ^^raU arrlas 



i_ 



caka. 



Plata 



to be I 



L 



J 



Whipped Cr««ii TrMt t09«t»»«r. 

• Start with th« flr>t thing you do. 

. T.n .boot MCh pTt of th, Tr..l th.t you put 
to9«ttMr. 

• Tan about th. parts In the order that you imist 
' . put thaai together, 

• te sure to use exact *«rd». 

. u.. -ord. I Ik. nsjct to join .a- of th. ..nteoc.. 
In your paragraph*. 

I t— Datctlptlon 

.klllt th« student -rit«» th« composition 
using .xposltory-r ting be • picture of • 

requested by the stimulus. *|'T^ "%7^! .^d. The order In 

,i^|e -hole -Ith Its P-'" '''Pji^;';;.o ^ ,Uly end gr.phic.lly 
^hlch the pert* ere ''/Kl„r (1 .7 . moving) perts 

ruirS'inJiuisiNr'tS M.^J-t-^ object. 

The serle, of Instructions - .nc.ud.n, f.,tur« J-^PJ 

production of :^^;:t;; it"Lse features, 

an approprlai* scoring keylhat evaluate. 
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«;<ioring Key 

.K. i.*w<« matrix consists of the features of 

5- I..!!; nforma'tlon about Individual component skills. It 
diagnostic information aoo writing score by assigning 

ru:L:l::i e<,urv^rntrnl:'g:od/average/unacceptable categories. 



Good Average UnacceptabI 
Scoring Criteria 

CONTENT: 

includes all parts of the Treat. 

Writes about the parts In the 
order In which they are put 
together. 

Uses precise language. 

Uses logical transitions. 

uses sentences that pertain to 
the iMin Idea. 

FOM: 

Uses correct qfm^r, 
uses good sentence structure. 

Capitalizes and punctuates 

correctly. 

Spalls correctly. 

writes legibly, -Ith appropriate 
margins and Indentation. 

J 

Scoring Guide 

constitutes a good, average, 
in the scoring key. 





W5 OPpr *iOOO (continued) 



Th* «kilU corresponding to each criterion are listed by 

The skil s correspw o y fDilow the criterion statenienl. The 

number w.thm that (I) must be employed 

trp^^ub he si:;" and (2) are designated as optical .assessment 

Ikil f at the irJdiate and earlier grade levels S^<'^»"5I9U felines 

fS o^e content characteristic and one form sK.I are ^-^"H) 

I^^oT Actual skill specifications include complete guidelines. 

COMTEMT -EXAMPLE: 

>;...d.sall parts of the Treat (i.e.. plate. cake stawberries. 
dipped cream, banana slices, whipped cream, cherry). 
(Skill GOe 2313I 

Good: All parts are included. 

Average: Most parts are Included. 

- Uoacceptabler Many parts are missing. 
rOM EXAMPLE: 

, oells correctly . (Skills Sc 1000. 2000^ 3000; Sv 1000. 2000. 
3b00. Coot; Sa lo OO, 2000; Ssp 2000. 3000. liOOO) 



Good: 

Average: 

Unacceptable: 

Comnent : 



Host words are correctly spelled. 

Several different words are misspelled. 

Many different words «re misspelled. 

The average and unacceptable ratings )^ 
-Spells correctly" criterion refer to '^^^"^ 
words" because many instances of mi sspel I Ing the 
same word should be evaluated as one misspelled 
word . 
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PROBLEMS/SOLUTIONS 



I. The Inclusion of distractor type A requi res! that the student distinguish 
between singular and plural possessive forms at this grade level. 
However, we identified the use of the apostrophe in plural possessives 
as a fifth-grade skill. Consequently, d i stractor A was revised so 
that the apostrophe preceded its correct position in the word (e.g., 
bir'ds). 



Distractor £ is grammatical and plausible. Thus the assessment emphasis 
is on the correct interpretation of the picture stimulus, rather than 
recognition of word order. However, the Items should not place undue 
emphasis on the student's ability to 'Vead" pictures. Therefore, we 
decided that distractor type C should clearly be implausible or ttie 
picture should be changed. 



3. 



The dialogue quotation is used as the stimulus i;ythis item^so that 
the exclamatory nature of the sentence can be Sriiggested by the soeaKer 
tag. However, if the tag follows the quotation, the item actualy 
assesses punctuation of dialogue quotations rather than terminal 
punctuation. Consequently, we specified that the speaker tag should 
precede the quotation in these Items; e.g.: 

Jan shouted, "The barn is on fire " 



The problem here is that distractor £ is inappropriate. This distractor 
foVms the word bare, a homophone for the corre^ct spelling. Whil^-the 
spelling of homophones is an important skill i'n and of itself, homophones 
should 'be assessed separately and should not be used as distractors in 
other spelling skills. Di stractor IC was revised to ere. 




53 



The correct response (choice £) requires additional context to 
establish its number. We decTded to delete plural nouns that retain 
the same form as the singular (e*g. , sheep , fish , deer) from this 
assessment* / 



As written, this item does not measure ability to discriminate 
rhyming words, but ability to discriminate final consonants. 
In order to match rhyming words, students must recognize that 
both the medial Sound and the final sound are the same. Thus one 
distractor should have the same vowel sound as the target word 
but a different final consonant; the other distractor should have 
a different vowel sound, but the same final consonant, Distractor 
B was revised to fun. 



This item assesses a reading rather than a writing ski ll-reading for 
the main idea. When students write, they don't construct a main idea 
after the paragraph. Rather, the writing problem is having a mam 
idea and fceeping to it. 

Directions: One sentence can be added to the paragraph 
because It fits the idea of the paragraph. 
Which sentence is it? 

Football players must be strong to play in a 
game. They must be able to knock other players 
down. Players must be able to run fast because 
fast runners can carry the ball to make points 
the game. 

A; Football players have to throw the ball hard. 

B. Footba-H players wear shirts with numbers on 
them. 

C. Some football players go to school to Isarn to 
play bal 1 . 

D. Basketball players run fast. 
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8. This Item type requires too much reading. The Item was revised so 
that the student only had to read the paragraphs once, selecting 
the point at which a new paragraph should begin; e.g.^ 

Directions: Read this story part. Where should a new paragraph 
begin? 

The bus returned to school In the afternoon, 
(a) Linda and RIcardo got off the bus and walked 
Toward the classroom, (b) ''Did you^joy the 
trip to the zoo?" Rlcarrfo asked, (c) Linda 
replied, "I sure did." (o) "My favorite part 
was the monkey hpuse because some of those 
monkeys act just like people,'* 



This Item has two problems: (1) the directions are not specific 
enough, and (2) the stimulus is inappropriate as it is written. 
Formal and informal language are relative to audience. Thus the 
directions should specify the audience. An appropriate writing 
situation in which a sixth grader might be required to use formal 
language wpuld be a letter to someone the student had never met. 

The use of gang in this sentence is unclear. If the writer is 
referring to a group of juvenile delinquents, then gang may be a 
more formal and accurate description of the students than any of 
the answer choices. This item was revised: 

Directions: The underlined word is too informal for most 

letters. Which word should you use if you wrote 
the sentence in a letter to someone that you have 
never met? 

A new bunch of Students will be coming to 
our school , 

A. gang 

B. group 

C. crowd 
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> INAPPROPRIATE ITEMS FOR ASSESSING WRITING SKILLS 

1 . Skill Statement 

Uses apostrophe in singular possessive forms. [Grade k] 

Assessment 

Sample Item 

Directions: Which word is right? 

Betsy has a pet bird. She wants to clean the 
cage, 

A. birds* 

B. birds 
@ bird»s 



Skill Statement 

Recognizes correct word order: subject-verb-object. [Grade 2] 

Assessment 

Sample Item 

Directions: Which is a good sentence that tells about the 
^ picture? 

[picture: girl running after a boy] 
^ Mary is chasing John. 

B. Mary John is chasing. 

C. John is chasing Mary. 
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3. Skill Statement 

uses exclamation point to end an exclamatory sentence. [Grade h] 

Assessment 

Sample Item 

Directions: Which Is the best mark to use in the blank? 
"The barn is on fire__" Jan shouted. 
A. , 

© • 

C. ? 

D. . 



k. Skill Statement 

Spells vowel-r pattern: /er/-are. /er/-ear. [Grade >] 

Assessment 

Sample item 

Directions: Which letter or letters finish the word? 
[picture: bear] 



b 



A. air 

(b) ear 

C. are 

D. er 
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Ski 1 1 Statement 

Distinguishes noun forms; singular/plural. [Grade 3] 
Assessment 

Sample I tems 

Directions: Which word means more than one? 

A. glass 

B. sleep 

C. house 
(d) sheep 



6. Ski 1 1 Statement 

Identifies words that rhyme. [Grade 2] 
Assessment 

Sample Item 

Directions: Which word rhymes with the name of the picture? 
[picture: nine] 

A. fire 

B. five 
(c) fine 
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Skill Statement 

Lirnits a paragraph to one main idea. [Grade k] 
Assessment 

Sample Item 

Directions: Which sentence tells the main idea of the paragraph? 

Football players must be strong to play in 
a game. They must be able to knock other players 
down. Players must be able to run fast. Fast runners 
can carry the ball to make points in the game. 

(a) It takes strength to play football. 

B. Running is an important part of football. 

C. Football is a fun game that everyone should play. 

D. Scoring points is important. 



8. Skill Statement ^ 

Uses dialogue for only one speaker in a paragraph. [Grade 5J 

Assessment 

Sample Item 

Directions: Whi ch conversat ion i s wri tten correctly? 

The bus returned to school in the afternoon. 
^ Linda and Ricardo got off the bus and walked 

toward the classroom. "Did you enjoy the trip to 
zoo?" Ricardo asked. 

"I sure did," Linda replied. "My favorite 
part was the monkey house," she said, "because 
some of those monkeys act just like peop e. 

Ricardo laughed and said, "Yes, you're right. 
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The bus returned to school in the afternoon. 
Linda and Ricardo got off the bus and walked 
toward the classroom. "Did you enjoy the trip to 
the zoo?" Ricardo asked. "I sure did," Linda ^ 
repilied. "My favorite part was the monkey house, 
she said, "because some of those monkeys act just 
like people." Ricardo laughed and said, "Yes, 
you're right." 

The bus returned to school in the afternoon. 
Linda and Ricardo got off the bus and walked 
toward the classroom. "Did you enjoy the trip to 
the zoo?" Ricardo asked. 

"I sure did," Linda replied. "My favorite 
part was the monkey house," she said, "because 
some of those monkeys act just like people." 
Ricardo laughed and said, "Yes, you're right. 



Skill Statement 

Uses appropriate formal/informal language. [Grade 6] 



Assessment 



Sample item 

Directions: The underlined word is too informal. Which word 
should you use? 

A new gang of students will be coming to our school 
A. bunch 
@ group 
C . c rowd 



'♦READING ASSESSMENT AND THE BILINGUAL TEACHER" 
NABE Pre-Conference Workshop 
Lai la Fiege-Kol Imann 
April 19, 1980 



La ; la F iege-Kol Imann 



INTRODUCTION 

Bilingual children and children whose mother tongue differs from 
English (NES/LES) usually take part in the reading instruct ion" i ntended 
for monolingual English speaking children. We know that the NES/LES or 
bilingual ch i 1 dren are learn ing to function in Eng 1 i sh and that they are 
acquiring reading skills at the same time. Yet, we often tend to forget 
this dual acquisition process, and expect these children to attend to 
reading instruction In the way typical monolingual children do. Not 
only are the bilingual or English language learners taught to read by 
monolingual reading methods; their progress in reading is assess ed with ^ 
the standard reading tests mandated for use in the public schpeffs today. 

"Bilingual and NES/LES children do not receive high scores on reading 
assessment." This is a comment we often hear and maybe even take as a 
fact of life without asking some hard questions about assessment in 
general, the instruments used in a given case in particular, the demo- 
graphic parameters on the populations under study, and so on. Reading 
assessment in general is presently undergoing scrutiny. For years, 
standardized achievement tests were used to measure progress in reading. 
Recently, there have been several studies to indicate that the skills 
tested on many standardized achievement instruments are on the 
periphery of the skills emphasized and practiced In instruction 
(Buchanan & Milazzo, 1980; Berliner & Rosenshine, 1976; Armbruster 
et al., 1977). Presently, reading assessment research is focusing on 
the link between instruction and assessment, i.e., in a normal 
classroom setting, progress in reading should be measured by instruments 
which reflect the skills children have had the opportunity to learn 
through instruction. 

At SWRL for many years now we have been doing research on the. 
nature of reading, instruction as Is evidenced in the contents of commonly 
used reading series and reading surveys. I will discuss very minimally 
the design and development of reading proficiency survey instruments, and 
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then I will spend some time explaining some recent reading proficiency 
data gathered from monolingual, bilingual, and NES/LES children. 

Design and Development of Survey Instruments 

One of the foremost aspects in the assessment of reading skills 
is the definition of the scope, sequence, and emphasis of reading 
skills within instructional practices. At SWRL we accomplished this 
by a lengthy process of cod i ng i nstruct iona I activities, grade-by-grade, 
and page-by-page. This computer-analyzed information shows the 
introduction, emphasis and maintenance of reading skills across grade 
levels, and it is being used as a basis for designing large scale 
Information systems measuring reading skill proficiency. Such surveys 
give a summary description of what has been accomplished in a school 
year and provide important indicators of what the student can be asked ... 
to do in the following year. 

I will discuss survey data from two information systems, developed 
in co-operation with two different school districts. The first set of 
data (Tables I and 2) will be used to look at the relationship of age 
and reading performance; the second set of data will be used to discuss 
children's language background and reading performance. 

/ 

It is often said that the child from a different language 
background is older than the monolingual child at a given grade level. 
In looking at third graders in a large urban school district, we found 
that B0% of the monolinguals are 9 years old, and ]h% are 10 years old; 
for bi Unguals the percentage differed about \0%: 68% were 9 years old, 
and 22% were 10 years old. This difference is less than one normally 
would have expected. 

■ Table I presents data on a fifth grade survey tailormade to the 
needs of an urban school district. The survey taps skills that the 
district considered important for the students to know before leaving 
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the elementary school system. The table indicates the breakdown of 
average percent scores by skill area and age group. As can be seen 
from the results, the scores go down as the grade levels progress. 
This decrease in scores Is regular and systematic except in one case, 
In the comprehension skill area 12 year old children perform worse 
than 13 year old children. From other, scores in Table 1 one would 
have expected the opposite to be true. 1; can not explain this 
discrepancy without further analysis of the data. 

Table 1 

Breakdown of average percent 
scores by skill area and age group; 
5th grade; N«2856 



Skill area 




10 


Age 
11 


12 


13 






88 


82 


6i» 


i»6 


Word Meaning 






Sentence Completion 




92 


86 


69 


52 


Verb, Noun Agreement 




91 


85 


66 


60 


Comprehens i on 




79 


72 


52 


60 


Table of Contents, Index 




92 


87 


80 


67 


Study 






66 


65 


Al phabet izAtlQJX, Glossary 


Si<,i l is 


87 


80 



Overa 1 1 



87 



81 



65~ 58 



The main conclusion to be draw,^' from Table 1 is the one already 
stated: as the age level of the students at a particular grade level 
increases the scores tend to decrease. If children from a different 
language background are older than their monolingual classmates, 
their scores also tend to be lower than the scores of their younger 
classmates.. Table 1 can also be used to show the difference in skill 
area scores between age groups. Comprehension skills are hardest for 
the 10-year-old students whereas word meaning and sentence completion 
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skills are more difficult for the 13-year-old students. For example, 
less than half of the 13-year-old students answered eacH word meaning 
icem conrectly while the performance score was €>0% for comprehen- 
sion skills. For the 10-year-old students word meaning scores (88%) 
were about \0% higher than the comprehension scores {73%)-* Comprehension 
skills represent global skills or skill aggregates which depend on the 
students' ability to mesh their own knowledge with the information found 
in text. It may be that the older students can more effectively use the 
contextual cues available in longer text and bypass same of the problems! 
they have in direct word mearting tasks.. On the other hand, the younger I 
students may benefit from the shorter and more direct tasks, patterned 
after exerc i ses ' found' i n their workbooks. 

The younger students do about 20% better in comprehension skills 
than older ones. Yet, this difference is much less than it is for 
word meaning, sentence completion, and verb/noun agreement skills 
where the difference in scores ranges from 30 to ^0%. It may be that 
the older students can take advantage of the general knowledge they 
have in comprehension skills which depend more on global skills and 
skill integration than the other skill areas, which include skills 
found solely in reading instruction. 

The 10-year-old students performed well in the study skills 
surveyed, and the 13-year-old students received their Highest 
performance scores in this skill area. This is what o|1e would have 
liked to expect since^he study skills surveyed cons is/ted of skills 
which children practice in reading but which are used also when 
reading in other subject matter areas such as social ^cience and 
health. They are organizational skills that require^less reading 
than what is demanded in other skill areas. j 

I 

Language Prof ic iency ,/ 

Table 2 shows the breakdown of'average percent?' scores by skill 
area and language proficiency for the sam^e survey which was discussed 



in the previous se'ction, the school district categorized the students 
as Fluent-English-Speakers, Some-English-Speakers, and No-English- 
Speakers- The No-English-Speaker label i s mi si ead i ng • I do not 
believe th^t there are any children in the public school system who do 
not speak and understand at least ml n imal Eng I i sh, and so I have 
labeled the groups Fluent English, Functional English, and Minimal 
English. The scores are as is to be expected. The less English one 
knows, the lower the scores. 

Table 2 

Breakdown of average percent 
scores by skill area and language 
proficiency; 5th grade, N=2856 



a 

Skill 


Language 

Fl uent 
Engl ish 


Proficiency 

Funct iona 1 
Eng 1 i sh 


Mi n imal 
Eng 1 i sh 


. Word Mean i ng 


86 


65 




1 Sentence Completion 


91 


77 


52 


Verb, Noun Agreement 


90 


70 


52 


Comprehens ion 


77 


61 




Table of Contents, Index 


91 


81 


64 


Alphabetization, Glossar^"^ 


85 


75 


53 



Overall 87 70 52 



The results presented in fable 2 offer other findings as well. 
The average scores of Functional English speakers are about 10 to 20?; 
lower thsn the scores of Fluent English speakers. The smallest differences^ 
can be found in the study skills, comprehension and sentence completion 
skills. These students tend to do less well with the word meaning and 



verb/noun agreement skills. These skills require lots of exposure to 
the language to be learned combined w!<th practice, and they often are 
stumbling blocks for second language learners. In study skills which 
depend more on organizing than extensive reading, these readers do 
relatively wel 1 . 

'in another study, the reading proficiency of 3rd and 6th grade 
students of an urban sc^ol district was surveyed. A brief description 
of the content of each survey is given in Tables 3 and k. Table 5 
presents the cumulative distribution of the results in 3rd grade; 
Table 6 lists the 6th grade results. In this particular study, we had 
a representative sample of the srchool district as a whole, and the 
results of this sample at S0% level are given in the right hand columns 
of Tables 5 and 6,- We -also had a representative sample of Hispanic 
students in the'district sample, and these students were classified by 
the district as follows: 

Engl i sh speakers 
Bi lingua Is 

Limited English speakers,' in English reading programs 
Limited English speakers, not in English reading programs 
Non-English speakers ' 
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Table 3 

Content of the Grade Three Survey 





Skill Area 


/ 


Number of ! tems 



Deco^^Fnrg" / 

(consonant digraphs; variant/ £, irregular 

vowe 1 patterns) 
Structural Analysis 

- (plurals; contractions; abbreviations; verb 

inflect ions; syl labicatioi/) ' 
Vocabu lary 

(sight words; definition^; word types) 
Comprehension 

(story detail;, sequence/ classification; prediction) 
Location/Study 

(parts of a book; alphabet ical order; directions) 



1 I 

16 

U 
12 

10 



Overall survey length/= 

/ 




63 i tems 


. / 1 

i 

Content of t| 


Table k 

he Grade Six Survey ^ 

\ ■ , — 





SkM) Area 




\ 
\ 


Number of 1 tems 











Decod i ng ^ . ' ' \ 

(vowel patterns; consonant digraphs) 
Structural Analysis 

(quantity prefixes) \ ' _^ 

Vocabulary 

(sfght words; definitions; 7iultip\le meaning words; 

antonyms, synonyms, figurative lartguage 
Comprehension 
/(main idea; sequence; cause/effect^ classify 

information;! quotation marks; conclusions; 

relevant/irrelevant information) 
Location and Study SkilJs \ 

(reference books; guide words; charts; graphs; 

naps; diagrams) 

Total Survey Length « 



15 
k 
U 

20 
21 

7^ i tems 



\ 



\ 



\ 



Table 5 

Performance Summary on the Grade 3 
Reading Survey 
Cumulative Distribution 



Student Classification 



Ski 1 1 Area 



District Overall (N-1773) 
Engl ish, (N-200) 
Bilingual (H»2k5) 
Ltd. English, Reading (N-112) 
Ltd. English, Not Reading (N-6l)| 
Non-English (N-}'*) 

District Overal 1 
Engl i sh 
Bilingual 

Ltd. English, Reading 
Ltd. English, Not Reading 
Non-Engl ish 



Decoding 



Structural Analys is 



Vocabulary 



District Overall 
Engl i sh 
Bilingual 
Ltd. Engl ish, 
Ltd. Engl ish, 
Non-Engl ish 



Reading 
Not Reading 



Comprehens ion 



District Overall 
Engl ish 
Bi 1 ingual 
Ltd. Eng^l-Uh, 
Ltd. Engl ish , 
Noii-Engl ish 



Reading 
Not Reading 



Location/Study 



Overal 1 



District 
Engl ish 
B i 1 ingua 1 
Ltd. English, 
Ltd. Engl ish , 
Non-Engl ish 



Reading 
Not Reading 



30% 1 


80^ 1 


70^ 1 


(TV 

o 


55 


1 

71 


80 


85 


55 


70 




82 


2k 


32 


A 


60 


18 


21 


3' \L 


36 


9 


(9) 


18 \t 


18) 


'«5 


55 


73 


78 


k2 


57 


70 


77 


'3 


21 


37 




]h 


16 


28 


33 


6 


9 


(9) 


(9) 


i»6 


60 ' 


6't 


71 


28 


i»8 


55 


68 


6 


10 


15 


27 


8 


1 1^ 


1 18 


1 22 


9 


(9) 


(9) 


(9) 




58 


73 


77 




61 


72 


78 


13 


18 


36 




12 


l^t 


2't 


26 


3 


6 


9 


(9) 


33 


'«8 


63 


72 


20 


'•9 


6't 


73 


11 


23 


32 


'•9 


1-2 


'7 


15 


29 


3 


(3) 


6 


12 



83 



76 



69 



75 



69 



71 



Table 6 



Performance Summary 
on the Grade 6 Reading Survey 
Cumulative Distribution 







Cumulative Distribution 


Student Classification 


Skill Area 


30% 




70* 


60* 


60* 


District Overall (N-I8OO) 

English (N-I98) 

Bilingual (N-270) 

Ltd. English, Reading (N-30) 


Decod ing 

Structural Ana lysis 


52 


66 
58 


70 
67 
16 


86 

8i4 


Sk 

69 
71 
70 
71 


District Overal 1 
Engl ish 
B i 1 ingua 1 

Ltd. Engl i sh, Reading 


Vocabulary 


23 
23 


(23) 
(23) 


60 
60 
50 


(60) 
(60) 
(50) 


District Overall 
Engl ish 
Bill ngua 1 

Ltd. Engl i sh, Reading 

District Overall 
Engl i sh 
B i 1 i ngual 

Ltd. Engl i sh, Reading 


Comprehension 
Location/Study 


13 
13 

17 
17 


20 
20 

33 
30 
3 


35 
3^ 
7 

^5 
10 


62 
6A 
17 

60 
56 
20 


Di St rict Overal 1 
Engl i sh 
Bill ngua 1 

Ltd. Engl i sh, Reading 




21 
20 
3 


3'» 
36 
13 

[ 


^•7 
kS 
19 


60 
58 
22 
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In looking at Table 5, one has to conclude that some students in 
the. Non-English and Limited-English groups are clearly mislabeled. If 
9 students out of 3^ in this Non-English group can score at 30% level in 
the word meaning skill, they have to be able to read and understand 
English. it seems that our classification procedures need redefining 
and reformation. 

The cumulative distribution scores show how well the students did 
at 90^, B0%, 70%, and 60% levels. For example, 55% of the bilingual 
third graders answered 30% of the survey items correctly in the decoding 
skill area, and 77% performed at 70^ or higher level in the same skill 
area (the percentage at the 70% level is larger than at the 30% level 
because it includes all students who performed at the 30% and 80% levels 
as wel 1 ) . 

In comparing the 3rd grade survey results between the district 
overall and the language groups (Table 5), It becomes clear than the 
English speaking Hispanic children ,and the bilingual children perform 
as well as the district does as a whole. This is true for all skill 
areas . 

English speaking Hispanic and bilingual children in the 6th grade, 
(Table 6) perform as well as the district as a whole in the decoding 
skill area , but in all other skill areas their scores are about 10°^ 
lower than the performance of the district as a whole. Thus, in the 
3rd grade, these children are performing at the overall district level 
whereas the performance in critical skill areas drops as the grade 
levels progress. There may be several reasons for this drop: more, 
older immigrant children, more mobility, more absenteeism at the higher 
grade levels, reading materials that are out of synchronization 
with the maturity level of the older students, and so on. For example, 
earlier in this paper i discussed the relationship between age and reading 
proficiency. Although the Hispanic English-speaking and the bilingual 
children performed as well as the district overall, they did not perform 
as well as the mono 1 ingual Anglo children did. On the average, the 
scores of the Anglo children were about ]5% better in x'le 3rd grade in 
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each skill area except vocabulary where the difference was about 20%. 
Among the 6th graders the difference was about 10^, except in the 
decoding skill area where there was no difference. The gap in the 
scores between the Anglo children and the Hispanic bilingual/English 
speaking children seems to be narrowing down by the 6th grade, which 
Is encouraging. 

D i scuss ion 

I have presented here some comparative performance data on children 
from different language backgrounds. The survey instruments used were 
designed to comply with specific school district needs. The survey 
items were developed to represent the kinds of items children see in 
their instructional materials. Although the survey instruments have 
been developed for the English speaking population, they can give us 
pertinent information about the relationship between the language 
background and reading proficiency." The data bring out more research 
questions than they answer: student, age, degree of proficiency, 
relationship of what is surveyed and what is actually taught, and so on. 

• The cut-and-dry classification into different language groups 
results in situations where children are grouped wrong. Presently 
there are no instruments or interview techniques on which the 
practitioner could rely in making more accurate decisions about 
student placement. 

It is sometimes assumed that monolingual English readers typically 
receive high scores and that these readers perform equally well in all 
skill areas. As these data show, the performance patterns among skill 
areas are mixed; they tend to vary from skill area to skill area. 
Readers usually do best in decoding skills whereas performance levels 
drop in vocabulary and comprehension skills (F i ege-Kol 1 mann , 1979, 1980) 
For instance, in the 6th grade d i str i ct csampl e (Table 6), m of the 
children scored S0% or higher in the decoding skill area, but only 
70^ scored at that level in the comprehension skill area. 
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The performance in all skill areas, and especially in structural 
analysis and study skills, is dependent on whether or not the children 
have received instruction -in the specific skills within the skill area. 
The children may learn the meaning of a word outside the instructional 
setting, but if they are not taught to interpret maps or diagrams or 
to understand the meaning of a quantity prefix, they can not perform 
well on items measuring these types of skills. 

It is usually the case that the skill area which is easiest for 
the monolingual English speaking readers is also the easiest for other 
language groups. Thus, all groups performed best in the decoding skill 
area whereas performance level drops for vocabulary or comprehension 
skill areas. ' " these studies, there was no one set of reading skills in 
which the basic differentiation between different language groups changed 
in any appreciable way. 

The discussion of reading proficiency in terms of achievement in 
various skill areas demonstrates the need for such an approach in reading 
assessment. The comprehensive nature of reading instruction is reflected 
best in a skill profile, rather than in a single performance score. Skill 
area scores will give a truer picture of individual achievement by 
pinpointing the skill areas where a student is progressing well and where 
more instruction is needed. Furthermore, the skill area information 
will give the teacher or practitioner a basis for moving the students on 
to a higher level of instruction within one skill area even though they 
may require additional help in some other area. If decoding scores are 
high and study skills low, the student should be able to move on to more 
advanced decoding while in study skills more instruction might be 
necessary. 

Another important issue critical to reading proficiency assessment 
is the match between what is taught and what is measured. If the match 
is good, the teachers at least know that the students have had the 
opportunity to learn the skills actually measured. 
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Often the skills included in a survey instrument are specified by 
a district or state continuum, rather than being based on reading instruc- 
tion and curriculum materials. If the survey instrument is designed to 
follow continuum objectives, the match between these objectives and 
instructional practices becomes critical; should the match be poor., the 
likelihood of low performance scores is high. 

These issues are of particular impor^tance when the language back- 
ground of the students is different from the mainstream Instruction. 
These students are extremely dependent on school learning since it is 
less likely for incidental learning to take place outside the reading 
classroom. 

In looking at reading performance, it is important to know (1) the 
instrument, its content and purpose; (2) the curriculum, so that the 
one being assessed does not become the victim between curriculum and 
assessment; and above all, (3) the child. No instrument will assess 
some aspects of the child's skill proficiency as accurately as the teacher 
who' interacts with him/her on a regular basis. The assessment instrument 
should be used to confirm teacher expectations especially when used with 
children form differ;fint language background. 
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SOME CONSIDERATIONS IN CONSTRUCTION AND ADMINISTERING 
LANGUAGE PROFICIENCY TESTS 

NABE Workshop Presentation 

I Bon i ta Ford 

•I 

1 

I 

Int roduct ?on _ ' 

\ 

In this presentation I intend to consider some of the problems which 
one is likely to encounter in constructing language proficiency tests, 
concentrating mainly on those having to do with Spanish testing. I will 
also consider some of the problems which may arise in administering 
these kinds of tests, and I wi 1 1 suggest some strategies for dealing with 
these problems- ^ 

What is a language proficiency test ? 

A language proficiency test is a tool which is used to determine 
the degree to which a child can understand, speak, read, and write in a 
given 1 anguage--say , English or Spanish. This is not to be confused with 
tests which have been constructed to determine language dominance (Does 
this person know one language better than the other?) and which involve 
testing in two languages at the same time. 

Why is it not advisable to translate language proficiency tests? 

Very often the need arises to fest language proficiency in a language 
for which there are no tests. The solution to this problem is either to 
construct a test for that language or to translate (adapt) an existing test 
from one language to the other. In the majority of cases people choose the 
second solution for a number of reasons--ease , speed, economy, etc. This 
practice, however, can result in a test which is not valid in the second 
language, that is, it does not test the same things that it did In the 
original nor does it test what it appears to. 



73 



Each language has Its own particular types of constructions, 
some of which. are mastered at an earJier age and some of which are not 
mastered until lat^r because they are more difficult. These structures 
may or may not be the same in both languages. In translating a language 
proficiency test one might 6nd up testing for a very easy type of 
construction^n one language instead of a difficult one, as was originally 
intended,, or pne might miss testing structures which are important in 
the second language. Consider, for example, the underlined word in 
the following sentences: 

\ John J_s_ a student . 
John \s_ happy. 

John hungry . ' * 

An\ English language proficiency test would be assessing a person's ability 
to\produce the appropriate form of the verb "to be*' and all three 

sentences would require the same form: "is." The same sentences translated 

\ 

into Spanish would require three different verbs: 

\ - 

ijuan es_ estudiante. 
\Juan est! contento. 
Juan t iene hambre. 

Likewise, a simple adjective in English^ such as\n "John is tal 1 ," Mary 

Is tall ," John and Mary are " tall ," will require three different forms of 

the adjective because of gender and number agreement in Spanish: Juan 

i ' 
es alto , MarTa es al ta , Juan y Marfa son altos . 

Construction A Language Proficiency Test / 

The purpose of a language proficiency test is to determine whether a 
student can understand a language anc| whether he/she can communicate in 
that language. We may also be interested ,irf finding out whether the 
student can not only communicate orally, but can also read and write in 
that language. A fremework such as j:he following can be used to guide 
us in constructing a language proficiency test: 



So 



' Comprehens ion '/ Product ion 

' . • Listening / • Sneaking 

• Reading y • Writing 

Within each of; these categories we might want to test different 
areas, such as grammar, vocabulary, decoding, spellinc, etc. We could , 
then go about constructing different tasks to assess €iach of these 
processes separately, whfle still keeping in mind that they are all 
interrelated. Based on my experience in constructing language pro- 
ficiency tests, I will point out some problems that might be encountered 
and recommendat ions'! for avoiding those, problems. 

Some Problems One Might Encounter in Constructing a Language Proficiency 
Test ^ 

1. Instruct ions . The instructions should never be more 
complicated that the task/ itself. Care must be taken 
to ensure that the vocabu/lary used in stating the. , 
instructions is as simpU as possible and never beyond 
the student's appropriate reading level. Instructions 
must be written in very/simple language and examples i 
should be provided, whe "e possible. 

2. Pictures and drawings . / Pictures or drawings are a very 
easy way to test a child's .vocabulary and they are often 
used in test ing-^comprehens ion , where the child is asked 
to point to the appropriate picture after a word is 

given in the target ^ 1 anguage . The problem often encountered 
here is that it is not always easy to find or draw a picture 
of what you want to test fo\ and this may constrain your , 
choice of items. For examp 1 e ,v the vocabulary item to be 
tested in the following pair of drawings was "ceja'*-- 
eyebrow . 




"figure 1 : "cej a' '--eyebrow. 



Unfortunately, most of the children to whom this drawing 
was shown could not come up with the correct word. The 
^ problem was not that they didn't Icnow the word "ceja," 
but that the drawing did not look like an eyebrow at 
aVI . When other people were asked what they thought the 
drawing represented, they said "a piece of yarn" or "a 
caterpillar." The drawing, in Figure 2 yielded better 
resul ts. 



3. Item Val idi ty . Does the item really test what you think' 
it does? Once you have determined what you want to test 
on a particular item, the next step is to construct a* 
cjuestion that will test it. You make up a question and you 
are convinced that people who know the answer will respond 
correctly, and that whoever doesn't respond correctly 
doesn't know the correct answer. It's quite possible, 
however, that even though you were guided by the best of 
intentions, the question you constructed did not really 
test what you thought It did. 

In attempting to test the phrase "dejar entrar al perro" — 
to let the dog in— we had to go through three different 
pairs of drawings before we came up with a good set of 
responses (Figures 3, ^ and 5). The object was to get the 
student to correctly Identify the picture that corresponded 
to the sentence, "The girl let the dog in the house." The 
two drawings' in Figure 3 were confusing beca.ise Figure 3a 
could be interpreted as "The girl wants the Jog to come 
out," or "The girl wants to go In the house." Figure 3b 
could be "The dog wants to go In the house." Neither drawing 
clearly elicits the correct response, "The girl let the dog 
in the house." Likewise, the pair of drawings in Figure k. 
Figure 5 shows the pair of drawings which produced the best 
results. Because therejs a logical relationship between 
the two plctures--the girl sees that the dog wants to come 
in the house, then she opens the door and lets the dog in — 
students are more likely to choose the correct drawing. 




' Figure 2 




Figure 3 




The recommendation I would make In order to avoid the kinds 
of problems pointed out in this example and in the preceding 
one is to test out the items with other adults as well as 
with children in order to find out whether you really are 
testing what you think you are. 

k. ProHuction Tasks in General . The trouble with trying to get 
chi Idren to produce wr.at they know is that, in most "ses, 
comprehension is being tested as well. It is very difficult 
to separate comprehension from production. For example, it 
you tell a child, "Write down the names of the objects you 
see in this room," the child may very well know the names 
of several objects in the room but has not understood the 
directive. 

Oral production tasks bring up another set of problems. How 
do you get a child to speak? If you are dealing with a shy 
child, if the child has not understood the task, or^if the 
task appears silly or senseless to the child, the child may 
not respond at al 1 . 

To. sum up the recommendations for avoiding the problems discussed 
above, in constructing a test one should: 

1. Make tasks as simple as possible. 

2. Use simple language and simple instructions. 

3 Avoid using words which may be culturally biased--you may be 
testing a child who speaks a different dialect and who may not 
understand the word you are using. 
k. Use aids whenever possible, such as pictures, drawings, tape- 
recorded sounds, etc. 
Some Problems one Might Enco u nter in Administering a Language Proficiency 
Test 

1. Ins tructions . Unless the instructions have been simply and 
clearly state d a child may not understand what is expected of 
him/her. Take the time to make sure the child understands 
the task to be performed. 

2 Reticence. A child may hesitate in responding or may simply 
refuse to answer. Children from different cultures have 
different ways of Interacting with teachers and adults m 
general. Don't assume that because a child does poorly on a 
task, that the child has language problems; the reason may not 
be language-related at all. 



One very common type of task for testing sound discrimination 
is to show a child a picture of an abject (e.g., a ball) 
and ask, 'Ms this a£alV'7 If the child can hear the 
difference between ''p*' and ''b,'' the correct response would 
be "No,'' or "No, It's a ball 

Do we assume that a child who answers "yes" has a language 
problem? Not necessarily. Maybe the child won't tell the 
tester when something is wrong because to do so would be 
considered rude in his or her culture. Maybe chil dren aren't 
supposed to correct their elders. It is obvious that, in 
any testing situation, one must be very sensitive to these 
kinds of cultural differences. 

3. Scoring. If you are not sure whether a response is correct 
or not, don't count it wrong. The child may have grown up 
in a different environment, and may know many things which 
you don't. A child, for example, who has grown up on a 
ranch probably knows much more about animals than you do-- 
different names for different types of animals, different 
types of feed for each animal, etc. Furthermore, many 
words and expressions have different meanings in. different 
cultures. People who have had contact with the Mexican 
culture know that a "taco" is a type of food made with a 
fried tortilla and stuffed with meat, lettuce, and 

tomatoes. In Argenthna, however, -and in other parts of | . 

South America, a "taco^' is the heel of a shoe; in Spain, 
'•taco" can mean "a bad word, an oath." When in doubt, 
suspend judgment until you can verify that the response 
is definitely incorrect. 



Summary 

When constructing items for a language proficiency test be aware of 
the problems involved. Don't assume that because you think you are testing 
for something in particular, you necessar 1 ly- are doing so. Take care to 
make written instructions as simple and as clear. as possible. Be aware 
of possible dialect differences and avoid using words that might have 
different meanings from those intended. 

When administering language proficiency tests, be sure that the 
child understands the instructions and feels at ease. Don't count things 

} 
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wrong if you are not certain that they are wrong; suspend judgement, 
if necessary, until you can verify the reporse. And lastly, remember 
that people have different accents and different ways of saying things 
depending on the area in which they grew up and the dialect which they 
speak. 
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ADAPTATION OF ENGLISH PROFICIENCY INSTRUMENTS FOR KOREAN 

Kenneth K. Kim 
NABE Preconference at NCBR, V'S/SO 

According to California state Assembly Bills 1329 and 65, NES/LES 
students should be assessed in their primary language proficiency, 
because the students' primary language is to be used ^ the language 
of Instruction In school. 

Many test developers have produced matching English and Spanish 
•language assessment instruments with the same rational^ and similar 
test format" Frequently such matching versions have^erits because the 
procedures of administration, the scoring, and the interpretation of the 
test results are the same or similar. One might also think that it is 
relatively easy to compare the results of the two tests to determine 
language dominance because of the similarity in test contents and 
methods. Perhaps all these considerations have prompted the trend to 
adapt existing English or Spanish proficiency tests for use in other 
1 anguages . > 

However, adaptation of the existing English or Spanish tests for^ 
non-lndoeuropean languages is not as simple as it might be thought 
because of the differences in the linguistic and orthographic systems 
of the languages involved. The purpose of this paper is to discuss 
problems arising from attempts to adapt some existing language assessment 
instruments for use in assessing proficiency in Asian languages. 
It Is not the concern of the presentation to assess the validity of the 
original versions of the instruments being discyssed. The concern is 
rather whetKer the original versions can be adapted without seriously 
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distorting the rationale, the objectives, and the validity appropriately 
credited to the original instruments. Although the discussions in this 
paper will focus on Korean, the same kinds of problems are expected 
to exist in other Asian languages and also in many non- 1 ndoeuropean 
languages. 

The four tests to be discussed are BINL (Basic inventory of 
Natural Language), BSM (Bilingual Syntax Measure), LAB (Language 
Ass,essment Battery), and LAS (Language Assessment Scales), all of 
wWich are approved by the State of California. The four tests are 
distinct from each other in a number of significant ways: (l) in the 
coverage of skills being tested; (2) method of response el i c i tat ion ; 
(3) scoring; and (4) theoretical ground for the method and coverage. 
A brief description of the four tests is presented in Table 1. Since 
the types of problems are different from test to test due to the 
differences in contents and method of testing, each test w.i 1 1 be 
discussed separately . 

BINL (Basic Inventory of Natural Language) 

' This test uses pictures to prompt the student to give an oral 
response. This method can be equally effective for any language. The 
major problem in adapting BINL for other languages is mainly in the 
analysis of the students' speech samples for the purpose of scoring. 
The BiflL scoring system refers to linguistic units such as word, phrase, 
clause, etc., as indicated on the sample BINL Scores Sheet in Table 2. 
The fluency score in column C ofT^e left hand side is the number of 
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Table 1 



Test Description 



Tests 



Rationale & 
Cnphasi s 



BINL (Basic Inventory of 
Natural Langu dge ) 

Linguistic complexity of 
oral samples collected in 
naturalistic speech settings 
best reflects children's 
language proficiency. 



BSH (Bi I ingual Syntax 
Measure) 

'Acquisition of morphological 
and syntactic items best 
reflects degree of language 
proficiency. 



Skills £ Picture-el i ci ted oral 

Nethods production. 



Elicit specific morphological 
and syntactic forms through 
structured conversat ion using 
cartoon type pictures. 



Levels 



K-adult 



Level I: K-2 
Level II: 3-12 



TttSting Time 
Scoring 



5 min. /child 



Counting linguistic units, 

such as words, phrases, clauses. 

Assign different weights to 

different units. 

Computation of counts and 

points to obtain proficiency 

score. 



10 min./chi Id 



Grammatical analysis of 
response. 

Count correct responses. 
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LAB (Language Assessment 
Battery) 



XAS (Language Assessment 
Scales) 



Language proficiency should be 

tested in four skill areas 

listening, speaking, reading , 
and wr i t i ng . 



Language proficiency test 
should ihclude i tMS from 
a 1 1 I ingtii St ic ski lis 
categories including 
d i scourse. 



L i s ten i ng/ speak i ng : your name, 
how old, body part naming, 
common object naming, sentence- 
picture matching. 
Read i ng : picture-word matching. 
Wr i t i ng : picture-word matching, 
grammar, writing mechanics. 



Level I: K-2 
Level M: 3-6 
Lrvel III: 7-I2 



Phoneme d i scrltinat Ion : 
minimal pairs (same-different). 
Vocabul ary : picture naming. 
Phoneme production : word/ 
sentence repetition 
Listening comprehension : 
point to correct picture of 
given sentence. 

Story retelling ; after hearing 
recorded stor/ retell it or 
answer prot« questions. 



Level I: 
Level II: 



K-5 
6-12 



Level I;. 5-IO min./chi Id 
Level n & 1 I I : k\ min. 



20 min. /child 



Count correct answers. 



Count correct answers.* 
Teacher's subjective rating of 
children's discourse production. 
Computation of conversed scores. 
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Table, 2 



Sample BINL Scores Sheet 



IfMNVIOUAL OUAL SCOHES SMttT OOS) 
•ASIC INVENTORY Of NATURAL LANGUAGE (WNL) 



Sludtnl'i Hmm 



I I rosT fisf 




. CMECpoim S f9ittm 
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wor^s in the sample transcribed in the same row. ..This fluency score 
along with the counts of other-units is used to compute the proficiency 
Jevel score. 

In English and Spanish, the words are simply orthograpjfi i.c units 
e>'cept for a few contracted forms. ThuS;, any teacher who can read and 
write these languages can count the number of words in the samples. 
Such an easy orthographic clue is not apparent in Korean. In Korean, 
an ortnographic syllable, which is- also a phonological syllable in 
most cases, is a 'cluster of a few letters. Each orthographic lyllable 
in a word con^a i ri i ng more than one syllable can be readily distinguished 
from other syllables "by a, short but recognizable vertical space between 
them, and individual words and phrases in a sentence are separated from 
other adjacent ^-ords o'^. phrases by a slightly wider space between them 
as i I I ustrated be low. ' I 

I. , ' ■ 

(1) a.' "IT. ^ . -X^IH ^J^,- ('i words) 
b.. Rijce, soup, and fish. {k words) 

,One might notici the parallelism in the Korean'and English examples 
above: that is,| vAjrds are those units that are separated from others 
by a clearly recjogn i zabl e space. However, the following example shows 

■ ■ • , ' ' ! 

that the situati|on is not so simple. • I 

(2) a. _^r£l_^4^fti_^f^- (Swords) , ' ' 

' Jchn-S the a^le-0 ate. . j^' ' 

b. jjhn ate the apple. words) . / 

The two nouns ir the above Korean example, that is, 'John' and 'apple/,' 
are Jmmed iatel y jfollowed by the subject and object case marking particles 
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as indicated by S and 0 respectively. Should these particles be counted 
despite the lack^of orthographic marking? The following example seeMS 
to suggest the answer. 

(3) a. ^lLii,^:3L ' 

a b c d e f I ^ ' 

b. '( Someone superior^ to the speaker ) must h ^ -e been caught ! ' 

r1 e a f 



, V ' b 

Although 'all the seven syllables in (3)a are written without any 
interVe^ng space of the kind shjfl^n in (l) and (2), the string 
is highly complex as illustrated in (3)b. This string is 
composed of (a) a verb root /^/, (h) passive voice infix V^*!/, 
(c) horiorific infix /M/, (d) past tense infix Z-^/, (e) /assumpt i ve 
inflx /^/, and (f ) exc lamatory sentence ending in the I'OW level of 
speech /^*+ /. Most of these^ftems wqfcid have to be expressed by 
independent words or phrases in English as illustrated in (3)b. The 
example clearly shows that the number of orthographic spaces is not a 
measure of the grammatical complexity of the string, nor does' it 
reflect the language proficiency of the speaker. For theSe^reasons , the 
root, the fnfi.xes, and the ending should be counted separately. i 

Likewise, the noun .part I c 1 es in Korean as in (2) frequently play 
/ I ' 

a role comparable to word order in English which determines the 

grammatical function of the words. Since the functions of nouns arc 

explicitly i nd icated by the particles in Korean,, the order, for example, 

between a subject afnd an object can be/ freely i nterchanged wi thout 

affecting the mearVing or gramma t i c^l i ty of the sentence. Considering 
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the important function of the particles, it seems to be only fair to 
count them as separate items. 

The difference in the units of counting may require redefinition 
of other larger linguistic units such as phrase and clause. The 
difference in the units is also likely to cause different ratios 
between the various units being counted (e.g., number of words or 
phrases within a sentence). All these, in turn, may require different 
interpretation of the score on Korean tests. 

In principle, procedures for analysis of the speech samples and 
for the computation of the scores can be developed. Cut whether the 
length of a sentence or the number of phrases and clauses within a 
sentence is an appropriate .measure of language proficiency in Korg^n 
is another matter. Without further studies on the relationship 
between language prof iciency and scores computed on the basis of the 
number of various linguistic units in Korean, a Korean version of 
BINL may be merely a mental exercise, the singnif icance of which is 
seriously in doubt. 

BSM ( Bilingual Syntax Measure) 

liome second language acquisition researchers share a hypothesis 
that there are universal cognitive mechanisms that govern the language 
acquisition processes. Dulay and Burt (197M and Bailey, Madden, and 
Krashen {]37k) tested the hypothesis by examining the acquisition order 
of so-called English "functors" in learners of English as a second 
language. Functors ai;e the f unct ion ^words that have only a minor role 
in conveying the meaning of a sentence, such as inflections of nouns 
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(John - John's.); verbs (jum£, jumps,., jumped), articles, auxiliaries 
(j£ going), copulas (He is.) , and prepositions. These functors can be 
easily elicited independently of .the topic of conversation, and it is 
easy to determine whether or not they are used correctly. The studies 
revealed that the acquisition order of the Eng 1 i sh. functors was largely 
the same regardless of the different first language background of the 
learners, including the native speakers of English. 

The BSM is based on the research findings about the acquisition of 
functors. if a student has acquired only the easiest functors, he 
or she is assigned a low level of proficiency. if the student has 
acquired functors higher in the hierarchical order, he or she is 
assigned an appropriately higher level. 

Now, what about a BSM for Korean language which, as we have seen, 
has a linguistic system drastically different from English or Spanish? 
Following are some factors that determine the acquisition order or 
complexity hierarchy among the functors. 

ik) Factors affecting the acquisition hierarchy of the functors: 

a. Concept complexity: e.g., simple vs. complex tenses 

simple indicative vs. complex aspects 

b. Regularity in morphological or syntactic variation: 
e.g., regular vs. irregular past tense 

c. Frequency of usage: e.g., progressive tense, pronouns, 

articles, prepositions 

All these factors vary from language to language. Table 3 shows English 
and Spanish functors used in BSh along with the information about 
whether there is an equivalent grammatical item in Korean for these 
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Table 3 

Structures Tested by BSM and Korean Equivalents 



Structures 



Engl i sh 



Word order yes 

Pronoun cases yes 

Progressive yes 

Copula yes 
Short/long plural yes 

Auxi 1 iary (be) yes 

Articles yes 



Regular/ yes 
irreg. Past 



Present indicative^ yes 



Possess i ve 



yes 



Conditional aux. yes 
Perfect aux. yes 

Present subjunctive 

Past subjunctive 

Past participle 
Inf ini ti ve 
Adjective gender 
Conj unct ion que 



yes 



no 



Span i sh 



yes 

yes 

yes 
yes 
no 

yes (estar) 
yes 



yes 
yes 



yes 
yes 

yes 
yes 
yes 



Korean 



yes 
yes 

(case particles) 

yes 

yes 

no 

no 

? (usage 
di f ferent) 

yes 



no 

? (identical 
wi th the case 
particles above) 

no 

no 

no 

no 

no 

? (usages different) 

no 

yes 



ERLC 



Yes: There is an equivalent grammatical form that can be tested. 

No: There is no equivalent. •^^t^^.i-^H 

?: There is an equivalent but it may not be tested for the indicated 

reasons. 
- — : Not tested. 
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functors, and if there is, whether acquisition of the 'functor can be 
used as a measure of proficiency in Korean. As shown in this table, 
there do not seem to be enough grammatical structures that can be used 
to assess proficiency in Korean. The conclusion is then that the 
adaptation of BSM for Korean language requires a complete redesign of 
the test based on the acquisition hierarchy among independently selected 
Korean functors- 

LAB (Language Assessment Battery) 

The LAB includes three sets of tests for three different grade 
levels, each of which includes four subtests for listening, speaking, 
reading, and writing. Level I questions are mostly general questions 
testing comprehension of simple sentences, naming of common objects, 
letter and word recognition, and writing alphabet letters, words, and 
simple sentences. No specific target grammatical structures are 
tested. Many questions thus can be adapted to Korean without affecting 
the objectives of the original test questions. 

Levels 11 and III contain, in addition to general questions like, 
those in Level I, questions designed to test specific sound contrast and 

grammatical items. 

Some problems that are likely to be encountered in the development 

of a Korean version of the test are as follows: 

(I) There are three different levels of tests. What are the 
vocabulary items appropriate to each level? Information is needed about 
the concept difficulty, and reading and speVling difficulty of words 
that children of different ages Cdi cope with. The test developer 
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should develop a Korean lexicon for this purpose or at least be 
familiar with the principles to be considered in the selection of words. 

il) Which sounds are easy or difficult to discriminate and which 
sounds are easy or difficult to produce? To answer this question, one 
should be thoroughly familiar with the methods of contrastive analysis 
and error analysis. Except for those items testing specific English 
grammatical structures, most items can be easily adapted to Korean due 
to the generality of the objectives of the items. 

LAS (Language Assessment Scales) 

The LAS assesses proficiency in five subskills; (l) discrimination 
of speech sounds, (2) naming common objects, (3) target phoneme 
production in words and sentences, (4) comprehension which is tested by 
matching a sentence played by a tape-recorder with the appropriate 
picture depicting the meaning of the sentence, and (5) listening to a 
recorded story and retelling the story. 

The t^st has only two pages and, except for the story retelling 
task, the procedures of administering and scoring the test are simple 
and straightforward. If the concept and structural complexity of the 
words and sentences presented in the test can be controlled in an 
appropriate manner, the test can be relatively easily adapted to any 
language without changing the format or type of contents being tested. 
The criteria and procedures for the selection of the sound pairs, words, 
and sentence structures in the original test are not stated clearly 
anywhere. Nevertheless, they are easy to infer because of tht 
conventional test format and the straightforwardness of the test 
objectives. In short, this test seems to be the easiest to adapt. 
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To conclude, the first two tests, that is, BINL and BSM, require 
further research on the relationship between language proficiency and 
the various linguistic units in Korean. Any imitation version without 
research evidence would seriously l^ck validity. On the other hand, 
the remaining two, that is, LAB and LAS, could be adapted with little 
difficulty due to the common conventional testing method and the greater 
generality of the test items. 

At any rate, should there be any pressing need for an instrument 
to assess a new language and should the decision be made to adapt an 
existing instrument for the new language, it is very important .that 
each of the original test items be analyzed and evaluated in terms of 
the specific objective of the test item to determine the feasibility 
of the adaptation. Furthermore, every effort should'be made to incorporate 
linguistic, psychometric and cultural considerations from people with 
appropriate training and background. 



WORKSHOP EVALUATION 



In orde|^ tQ evaluate the workshop, a questionnaire was distributed 
after the final presentation. Of tlie seventeen participants, fourteen 
cor^pleted the questionnaire. Six questions were asked, four of them 
forced-choice and two open-ended. The qxjestions were designed to seek 
information about workshop effectiveness and usefulness and to provide 
opportunity for the participants to suggest ways to improve the workshop. 
The evaluation form is reproduced on the following pages and an analysis 
to each question follows. 
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NCBR/SWRL 
NABE Workshop 
April 13, 1986 



WORKSHOP EVALUATION 



We need your frank and constructive feedback to be able to plan 
other Workshops. Please take a few minutes to fill out this form. 
Thank you. 

1. This Workshop helped me by: 

synthesizing and organizing ideas 
reminding me of ideas I have neglected 



clarifying some of my ideas 
"""confirming my ideas and techniques 



changing my ideas , ui 

'redefining some ideas I had to make them more va uable 



"giving me new ideas, i.e., more approaches than I was 
aware of before 

providing increased understanding of new approaches 

giving me ideas I can put to use 
l^motivating me to find out more^ 
jnotivating me to try out some ideas 

other ■ - 



2. In terms of practicality, the ideas discussed were: 

verKpractical (easy to use and in tune with schools) 

practibi] (able to be used with a minimum of adjustment} 

somewha tVact i ca 1 (able to be used with change and 

considerable effort) . rr 
impractical (would require too much change and effort 

to be rewarding) 



3. Do you feel this was a good use of your time? 



_def ini tely yes 
_yes 

_somewhat 
_no 

def i ni tely not 
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Workshop Evaluation 
Page 2 



i*. Here are some ways, to improve this Workshop: 
(Brai nstormi ng) 



(Sharing ideas) 



(Presentations and demonstrations) 



5. Overall I would rate the Workshop 

excel 1 ent 

^very good 

good 

_f a i r 

poor 



6. Comments: 
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Question ^1 . , 

The participants were asked to evaluate ^^he usefulness of the workshop 
by checking the statements which described the ways in which the workshop 
was of benefit to them. Two of the participants checked all of the options 
in this question. Of the remaining participants, seven responded that 
the workshop was useful in "synthesizing and organizing ideas." The next 
most frequent response (six) was in "clarifying some of my ideas" and in 
"providing increased understanding of new approaches." Participants also 
responded highly on "reminding me of ideas I had neglected"; "redefining 
some ideas ! had to make them more valuable".; and on "motivating me to 
fine out more" (five each). Of the participants who did not check all the 
options, four expressed that the workshop was helpful in "confirming my 
ideas and techniques" and in "giving me ideas I can put to use." The 
options which received the lowest responses were "changing my ideas"; 
"giving me new ideas, i.e., more approaches than I was aware of before"; 
and "motivating me to try out some ideas" (three each)* None of the 
participants wrote in the space provided for comments. Participant 
responses and the response totals are presented in Table 1. 
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. Uble 1 
Ways Workshop was Useful 



Participant 



1 2 3 A 5 6 7 8 9 10 11 12 13 1 A Total 



Synthesizing and : v x x 9 

organizing ideas xxxxx xx . a 

Reminding me of ideas y x 7 

I had neglected x x x x x ^ x x 

Clarifying some of g 
my ideas x x x x x x . x x 

Confirming my ideas ^ ^ 
and techniques x x x x 

X 3 

Changing my ideas x x 

Redefining some 7 

ideas I had x x x x x x x / 

X 3 

Giving me new ideas x x 

Providing increased v v x x x 8 

understanding x x x x x x 

Giving me ideas I v x x x 6 

can put to use x x ^ 



Motivating me to find 
ouJ: more 



XXX 



Motivating me to try ^ ^ ^ 

out some ideas x x ' 



Other 
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Question #2 

/ In this forced-chvoice question, participants were asked to 
indicate if the ideas discussed were very practical, practical, somewhat 
practical, or impractical. Six out of the fourteen responded that the 
ideas were very practical (easy to use and in tune with schools). 
Four rated the ideas discussed as somewhat practical (able to be useH 
with change and considerable effort), and three responded that the ideas 
were practical (abVe to be used with a minimum of adjustment). One 
participant did not respond to this question at all. Table 2 gives the 
responses and the totals of each response. 



Table 2 



Responses To Rating Question 



Very Practical 
Practical ' 
Somewhat Practical 
Impractical 
No Response 



XXX XX 



P.rH.inants ' 1 2 3 ^ 5 6 " 7 8 9 1 0 1 1 1 2 1 3 U Total 



X XX 



X 6 
X X . 3 

k 
0 
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Question #3 

In this forced-choice question participants were asked whether they 
felt that the workshop was a good use of their time or not. Seven of 
the fourteen participants responded with "definitely yes" that the 
workshop was a good use of their time. Five more participants responded 
"yes," and the rest responded "somewhat." None of the participants 
indicated that the workshop was not a good use of their time. Table 3 
gives the resjjonses by participant and the totals.,^ 

Table 3 >, 
Workshop Was Good Use Of Time 



Participant 



1,2 3 4 5 6 7 8 9 10 11 12 13 1^ Total 



Definitely "Yes" xx x xxx x7 

Yes ' X X X . X X 5 

X X 2 
Somewhat . » a 

V 

0 

No . ; 

Definitely "No" " ^ 
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Question 

This question gave the participants an opportunity to suggest ways 
to improve the workshop. Nine participants chose to respond to this 
open-ended question. The comments ranged from "No need to change format" 
to suggestions for improving the ^Workshop by changing the format, to 
something such as "Panel d i scuss Ion fofmat." Other suggestions recommended 
more participation by the participants, such as actual test administration 
or test construction exercises. The responses are quoted verbatim below: 

• No need to change format 

• More participatic^n by participants 

• More background into research 

• Not enough time 

• Provide more actjjal examples; more absolute solutions 

• Panel discussion format 

• Working through a problem. Experience actual development 
of simple project 

• Discuss strengths and weaknesses of specific tests 
regarding criteria discussed 

• Should be in various languages 

• Provide participants opportunity to give the tests 

• Valid and reliable assessment instruments 

• Presentations on English literacy could have drawn clearer 
discussion and conclusions regarding bilingual pupil 
assessment 



Question ffS 

In this forced-choice question participants were asked to select 
an overall rating for the Workshop. Five participants rated the Workshop 
..Good." Four participants rated it as "excellent" and another four 
as "Very good." A rat i ng of "Fa I r" was g i ven by one part i c I pant . The 
participant responses to the question are given in Table i*. 

/ Table k . 

Workshop Rating 



Participants 'I 2 
Excellent x 
Very good x x x x 

Good ^ 
Fair 
Poor 



X X 



3 /, 5 6 7 8 9 10 n 12 13 1^ Total. 

X 

k 

K 5 



XXX 



Hi 
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Question #6 

Finally, participants were given space to make addi tional corameats 
about the Workshop. Seven participants responded. Each response is 
given below verbatim: 

• It was well-planned and most helpful. 

• Investigation should be done to find out what kind of language 
assessment tools are being used in Mexico and the rest of 
Latin America. 

• Very good. Pase muy buena experiencia. Espero que sigamos 
teniendo estos workshops. 

• Workshop was helpful to me to the extent that i will be 
scrutinizing the assessment instruments I use with my 
children. Presenters should have given us a list of test 
instruments they consider better or most appropriate for 
language proficiency determination. 

• It appears that bilingual testing is still in its infancy. 

• All presentat-ons were well prepared, clear, short, and to the 
point. Thank you for the excellent service! 

• Research is of course necessary, however, should lead to the 
development of practical assessment instruments. These 
instruments are what is now needed to accurately assess NES/LES 
students. 
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